Abstract:

This research paper presents a comprehensive analysis of the factors influencing the adoption and user satisfaction of diabetes mobile health apps. This work evaluates six (6) machine learning regressors and employs Ordinary Least Squares (OLS) multiple regression, including a polynomial regression extension, for hypothesis testing. It is important to note that this research is novel in its use of various machine learning regression models to explore the determinants influencing the adoption of diabetes mobile apps, while also considering the user experience journey.
By employing machine learning algorithms, particularly a stacked model with ridge regression, the study identifies developer reputation, usability, update frequency, and cost as significant determinants of app downloads; a proxy for adoption rates. The Stacked Model's superior predictive accuracy is evidenced by its result of achieving the lowest RMSE (0.4212) and highest adjusted R² (0.9586), outperforming other models such as Random Forest and XGBoost. Additionally, user feedback analysis sheds light on the varying levels of user dissatisfaction across different UX stages, with the highest discontent observed during the churn stage, despite fewer reported pain points.
The study's findings are supported by permutation feature importance analysis, F-statistics, and p-values. Key insights reveal that while update frequency may not greatly influence downloads, ease of use and developer reputation significantly impact user adoption rates. Furthermore, the research delves into business models, revealing that 'Free' and 'Freemium' models are particularly effective in the app market, while regional factors, such as those about Taiwan, also play a crucial role in adoption.
Recommendations from the study stress the importance of addressing technical glitches, enhancing connectivity and integration with health devices, providing educational content, and focusing on user-centric design. Finally, the paper underlines the need for such a complex approach in app design that puts users’ requirements first and proposes to improve the predictive modelling for real-time solutions.

1.0 Introduction

Diabetes is a prevalent global disease that is often characterised by high blood sugar levels. There is a statistical projection that people with the disease will boost from 425 million in 2017 to 693 million in 2045 (IDF, 2017). The inability of the body to produce insulin is responsible for Type 1 diabetes. The inability to find a cure for Type 1 diabetes necessitates dependable insulin administration for patients. After diagnosis, regularly monitoring blood sugar, administering insulin, and making lifestyle changes become necessary for management. The swift progress in Information Technology (IT) has facilitated the creation of mobile apps that can be downloaded.
With the potential to aid in condition management, mobile apps benefit people with diabetes. With a focus on blood sugar management and insulin monitoring, these apps deliver a seamless experience.
Mobile health (mHealth) technologies have emerged as a promising tool for diabetes management, providing patients with personalised support, education, and monitoring. Evaluating the quality of mobile apps from a user perspective is best done by analysing user feedback (Haoues et al., 2023). Adoption can be measured by the frequency of usage of the app and the extent to which it forms their routine and lifestyle. The number of downloads of an app indicates the popularity and acceptability of mobile applications by users (Finkelstein et al., 2017). To align with existing literature, downloads will be used as a proxy for adoption in this study.
This study bridges this gap of Zhang et al., 2023 by using regression analysis to identify app features that significantly predict app success (downloads), it’s essential to consider the contextual relevance of these features. The above-mentioned existing studies did not delve deeply into the specific features that drive adoption. Addressing this gap by focusing on diabetes-specific features could enhance the field. Furthermore, my work aims to extract user sentiments from reviews; existing research by (Eng & Lee, 2012) did not thoroughly explore the sentiment dynamics related to diabetes mobile apps. Investigating how users perceive features (both positively and negatively) and how these sentiments evolve could provide valuable insights. While content analysis is a common approach, studies like Krishnan and Selvam (2023) did not explicitly focus on pain points in the user experience journey. This research work identifies specific pain points related to diabetes management (e.g., functional utility, usability, privacy concerns) that would be crucial for targeted improvements. It is also pertinent to note that these studies lack a comparison between different models. Also, it evaluates about six (6) machine learning models and compares their predictive capability, efficiency, and relevance. While some of the studies discuss determinants of adoption, actionable insights for developers are often missing. This work would provide specific recommendations (e.g., feature enhancements, keywords for promotion) to bridge this gap and guide diabetes m-health app development effectively.

1.1 Aim and Objectives

The aim of this study is, "How do specific features, as identified through content analysis and regression analysis, influence the adoption of diabetes mobile apps, and what actionable insights can be derived?
This aim shall be achieved by attaining the following objectives:
Determine which app features significantly influence the adoption of diabetes mobile apps, focusing on downloads as a proxy for adoption.
Analyse user feedback to identify and understand the impact of different pain points on user dissatisfaction levels across various UX stages.
To compare different regression models in terms of their predictive efficiency, and relevance in identifying factors that influence the adoption of diabetes mobile apps.

1.2 Research Questions

What specific features of diabetes mobile apps significantly influence the adoption of diabetes mobile health apps (measured by downloads)?
How does dissatisfaction vary across distinct user experience (UX) stages, and what are the characteristic pain points influencing dissatisfaction within each stage?
What are the comparative predictive efficiencies and relevancies of different regression models in determining the factors that influence the adoption of diabetes mobile apps?
Research Hypothesis

Alternative Hypothesis (H1):

Features within diabetes mobile apps, when identified through regression analysis, significantly influence user adoption of diabetes mobile apps.
Null Hypothesis (H0):
Features within diabetes mobile apps, when identified through regression analysis, do not significantly influence the adoption of diabetes mobile apps.

2.0 Literature Review

2.1 Determinants of App Adoption and User Engagement

Diabetes is a global health concern, and mobile health apps have emerged as potential tools for diabetes management. Studies have indicated that affordability is a major concern for patients, and higher costs can deter app usage (Hou et al., 2018). Well-designed interfaces that are user-friendly can enhance user experience, increase satisfaction, and encourage adoption (Oughton, 2022). Krishnan and Selvam (2019) found that app rating, number of installs, app description length, and keywords like “free” and “health” are positively associated with app downloads. However, the study might not deeply investigate diabetes-specific features or user sentiments. Mehraeen et al. (2021) found that blood sugar tracking, medication management, educational resources, and social support are important features of a diabetes management mobile app. However, their work lacks a focus on the pain points faced by users along the user experience journey as they use the diabetes management mobile app. This is a gap this study would bridge.
According to a study by Hou et al. (2018), younger adults showed higher responsiveness to mobile phone applications designed for self-management, reflected in their lower HbA1c levels in comparison to older adults. According to Bonoto et al. (2017), apps tailored to distinct demographic groups, like those with type 1 diabetes, yielded better glycaemic control. Furthermore, patients were more likely to use DSM applications if they routinely checked their blood glucose levels (Trawley et al., 2017) and engaged in regular physical activity (Ernsting et al., 2019). Patients who do not have diabetic problems and whose diabetes is under control are less likely to use DSM apps (Jeffrey et al., 2019). According to Peng et al. (2016) and Surkan et al. (2019), patients are more likely to use DSM apps if they help them communicate with HCPs and other patients, are aesthetically pleasing, are simple to use , and are easy to understand (Scheibe et al., 2015); guarantee privacy and accessibility (Torbjørnsen et al., 2019); Furthermore, as stated by Tanenbaum et al. (2016), patients' privacy and security are ensured; they offer immediate feedback (Pludwinski et al., 2015), personalised information, and facilitate goal-setting (Brandt et al., 2019); they are affordable (Scheibe et al., 2015), and they are available in the patients' native language (Kabeza et al., 2019). If patients encounter technical issues that result in frequent app crashes, they are less likely to use DSM apps (Kayyali et al., 2017).
Arnhold et al. (2019) used multiple regression analysis to decipher the relationship between usability and app functions. They found that usability is positively correlated with recipe suggestion and communication function. Similarly in diabetes treatment, Krishnan, G. and Selvam, G. (2019) used multiple regression analysis to identify success factors in diabetes smartphone apps. They also found that content review added value by providing classifications for diabetes mobile apps and will aid the patients in understanding the features available in the diabetes apps.
Findings from Husted et al. (2018) indicated that individuals with diabetes who used a smartphone to manage their condition experienced greater control of their condition and closer connections to their medical professionals. Furthermore, Garg et al. (2017) found that adults who used diabetes mobile apps to track their blood sugar levels reported feeling more confident in controlling their condition.
According to Sun et al. (2019), older Chinese patients with type 2 diabetes who employed a mobile app to interact with their healthcare providers and peers exhibited greater adherence to their treatment regimen. A study by Hou et al. (2018) uncovered cost as a significant obstacle to the adoption of diabetes management apps. Specifically, the high cost dissuaded patients with diabetes from utilising mobile apps, while low-cost alternatives proved more appealing.
The studies described above have shown that there is limited understanding of the factors which could foster engagement with apps to aid adoption. Hence, it is important to build upon previous research by examining the perception of a diverse range of people with diabetes; that is, both current users and non-users, residing in diverse locations, about the usability and functionality of diabetes apps to support their healthcare and factors
for usage over time.

2.2 Analytical Methods Used in Examination of Determinants Influencing Diabetes Mobile App Adoption

Krishnan and Selvam (2019) used content analysis of mobile health apps and regression analysis to identify the factors that are associated with app downloads. Their work primarily used only a statistical multiple linear regression model but did not take into cognisance the non-linearity of the features and comparison with other machine learning models. Also, content analysis was only done to classify features but did not harness the pain points of users, as their work failed to mine data on user reviews to carry out this analysis. Mehraeen et al. (2021) used a qualitative method, systematic review of literature, and interviews with patients and healthcare professionals to identify the features that are important for a Type 2 diabetes management mobile app.
Moreover, this research delves into extracting user sentiments from reviews, expanding upon the existing work by Eng and Lee (2012), who did not comprehensively investigate the sentiment dynamics associated with diabetes mobile apps. It is also pertinent to note that these studies lack a comparison between different models. This work evaluates about six (6) machine learning regressors and utilizes the polynomial OLS multiple regression model for hypothesis testing.
The studies are related to the topic of diabetes management apps, but they have different focuses and methods. For example, Zhang et al. (2019) explored factors influencing patients’ intentions to use diabetes management apps, while (Eng & Lee, 2012) reviewed the current state of mobile health applications for diabetes and endocrinology.
The studies used different theoretical frameworks and models to guide their research. For example, Zhang et al. (2019) used the Unified Theory of Acceptance and Use of Technology (UTAUT) model, while Alaslawi et al. (2022) used the Technology Acceptance Model (TAM) and the Diffusion of Innovation (DOI) theory. The studies had different data sources and analysis methods. Their works both lack a detailed exploration of how different app features specifically impact users' adoption of these apps and limited analysis on the direct correlation between specific app functionalities and improved diabetes care outcomes.
Krishnan and Selvam (2023) used a quantitative method, content analysis of mobile health apps, and regression analysis to identify the factors that are associated with app downloads; their work Primarily used only a statistical multiple linear regression model,, taking into cognisance the non-linearity of the features and comparison with other machine learning models. Also, content analysis was only done to classify features but did not harness the pain points of users, as their work failed to mine data on user reviews to carry out the content analysis. Furthermore, app rating, number of installs, app description length, and keywords like “free” and “health” are positively associated with app downloads. However, the study might not deeply investigate diabetes-specific features or user sentiments. Mehraeen et al. (2021) used a qualitative method, systematic review of literature, and interviews with patients and healthcare professionals to identify the features that are important for a mobile app for self-care of people living with Type 2 Diabetes. Mehraeen et al. (2021) found that blood sugar tracking, medication management, educational resources, and social support are important features of a mobile app for the self-care of people living with Type 2 Diabetes. However, their work lacks a focus on how users perceive the integration of these features into their daily routines and the challenges they face.

Table 1: Review of related works that fully integrate the above literature that has been critically assessed, their research focus and key findings. This reflects a comprehensive knowledge of the domain influencing the adoption of diabetes management apps.

The related studies described in Table 1 have shown that there is limited understanding of the factors that influence adoption of diabetes management mobile apps as well as a lack of the utilisation of machine learning regression models. Hence, it is important to build upon previous research.

3.0 Methodology

3.1 Dataset:

This study employs a comprehensive dataset that provides an insightful look into the mobile application landscape. Initially consisting of 86 applications, the dataset was substantially expanded through bootstrapping to encompass 2,000 entries. This enhancement was instrumental in ensuring a robust and comprehensive analysis. The dataset, however, does present instances of missing values in several features, notably in 'Short Description', 'Country', 'Website', and 'Price Currency'. These missing entries pose challenges but also opportunities for data imputation and robust handling strategies, ensuring the integrity and usefulness of the analysis.

Data Augmentation, Collection and Quality

A key augmentation to the dataset is the addition of the 'User Reviews' column. This data, sourced through web scraping techniques utilising tools such as Appbot and GooglePlayScraper, offers valuable qualitative insights into user feedback from the Google Play Store. Appbot is a tool designed for aggregating and analysing user reviews from major app stores. It specialises in extracting reviews and applying natural language processing (NLP) techniques to categorise sentiments and identify key themes. Also used was GooglePlayScraper, which is specifically tailored for scraping data from the Google Play Store. It efficiently extracts detailed information about apps, including user reviews, which are vital for understanding public reception.
The initial step involved identifying the specific mobile applications from our dataset for which user reviews were to be extracted. For each app, parameters such as the app identifier, the required number of reviews, and the time frame for the reviews were set. These parameters were aligned with the objectives of the analysis to ensure relevance and consistency. Using Appbot, user reviews were pulled based on the app identifiers. Appbot’s capability to analyse sentiments within the reviews was particularly beneficial, as it provided an additional layer of data categorisation.
GooglePlayScraper was employed to extract reviews directly from the Google Play Store, focusing on recent and relevant reviews that matched the study's timeframe. The reviews were then systematically integrated into the existing dataset. This involved mapping each review to its corresponding application based on the app identifiers, ensuring accurate and relevant data augmentation.

3.2 Data Pre-Processing

3.2.1 Textual Pre-processing

User reviews from selected apps were extracted using a Python scraper and App-bot and stored in a CSV file. The dataset underwent Python-based preprocessing, including cleaning and Unicode normalisation with Python NLTK (such as splitting text and handling punctuation), removal of common stop words using NLTK's list, and lemmatisation to group words with similar meanings using NLTK and WordNet, as seen in Figures 1 and 2 below.

Figure 1: Word Clouds Representing User Feedback on Diabetes Management Apps. The visualization displays prominent terms extracted from user reviews.

Source: Personal Jupyter Notebook.

Figure 2: Flowchart of User Review Analysis Pipeline: Data Augmentation, Preprocessing, and Sentiment & Topic Evaluation

Source: Personal Jupyter Notebook

3.2.2 Numerical Data Preprocessing

In the data preprocessing, the percentage of missing data was analysed to be 13.50% in 'Website', 10.10% in 'Country', 96.00% in 'Price Currency', 14.80% in 'Short Description', 1.25% in 'Developer', and 1.25% in 'Version'. Mode imputation was used for 'Country' and 'Developer' to replace missing values with the most frequent ones. For the 'Downloads' field, KNN imputation filled in missing values based on similar entries, and the data was transformed into midpoint values with a log transformation to normalise the distribution and reduce skewness. These measures ensure data integrity and analysis accuracy. The remaining columns, such as ‘short description’, ‘website’ and ‘price currency’, were dropped, as they were not useful for the study.

3.2.3 Feature Selection Comparison & Choice:

Spearman's rank correlation coefficient, as shown in figure 3 below, was used instead of Pearson's due to the skewed distribution of key variables like 'Downloads' and 'Price Numeric'.

Figure 3: This visual represents a Spearman correlation matrix, showcasing the relationships between different features of the dataset.

Source: Personal Jupyter Notebook

3.2.3.1 Comparative Analysis of Feature Selection Methods: K-Best vs. RFE:

The process of feature selection plays a key role in the performance of machine learning models. It involves choosing the most relevant set of features to enhance predictive accuracy. In this analysis, two popular feature selection methods, K-Best and RFE (Recursive Feature Elimination), were compared to choose the most suitable for this study, with a particular emphasis on their impact on predictive accuracy, as measured by Root Mean Square Error (RMSE). RFE outperformed K-Best in terms of lower RMSE and higher R-square values across various models. This makes RFE the preferred choice for feature selection in this project, aligning to optimise predictive accuracy.

3.2.4 Hypothesis Testing:

In the polynomial extension of the ordinary least squares (OLS) model used for this study, statistical significance will be evaluated using p-values. A p-value less than the predetermined alpha level of α=0.05 will lead to the rejection of the null hypothesis, indicating that there is a statistically significant relationship between the app features and user adoption of diabetes mobile apps.

3.3 Model Choices:

Post feature selection, several machine learning models were initialised for the study, including RandomForestRegressor, XGBRegressor, DecisionTreeRegressor, KNeighborsRegressor, SVR and Stacking Ensemble.

XGBoost Model:

XGBoost (XGB) is a powerful ensemble learning method that was proposed in a study by Chen and Guestrin (2016). It yields greater computational speed than existing boosting algorithms such as AdaBoost (Sagi and Rokach, 2018). It introduces regularisation parameters to reduce overfitting.

K-Nearest Neighbours:

The K-nearest neighbours (K-NN) algorithm is a simple, non-parametric machine learning algorithm. For regression, the K-NN algorithm predicts the target variable for a new data point by averaging the values of the target variable for its K nearest neighbours in the training data.

Support Vector Regressor (SVR):

SVR (Support Vector Regression) is a powerful regression model that has capability of addressing non-linear relationships between input variables and the target variable. It achieves this using a kernel function, which transforms the data into a higher-dimensional space.

Random Forest Model:

Random forests (RF) are a type of ensemble algorithm that aggregates the predictions of several decision trees to create a higher predictive capacity model (Breiman, 2001). Each decision tree in forest is built independently by using a subset of the training data and a random subset of the features to mitigate the risk of overfitting.

Decision Tree Model:

A decision tree is a machine learning algorithm that is constructed from a given initial feature, feature split, and decision threshold to segment the data into smaller subsets. Their representation in terms of IF–THEN–ELSE rules makes them relatively interpretable.

Stacking Ensemble:

Stacking is an ensemble learning algorithm that combines multiple base learners to create a more accurate and robust model. Wang et al. (2020) demonstrated its strength by using heterogeneous base models and a meta-learner to enhance predictive performance. To maintain interpretability, variable permutation importance is applied to the stacked model (Barton and Lennox, 2022).

Table 2: A table showing the stacking model base model and hyperparameter. The hyperparameter was derived from grid search.

The models had their hyperparameters tuned using grid search, as seen in Table 2 above. This was done to prevent overfitting.

3.4 Model Training

Training was conducted with Python 3.4 and Scikit-learn 1.0.1 on an Intel Core™ i7-6500U CPU @ 2.50GHz.

4.0 Evaluation Metrics

Adjusted R²

In the case of nonlinear models, the R2 is inappropriate for the demonstration of performance (Spiess and Neumeyer, 2010). This is because as more features are introduced to the model, its value increases even if the features added to the model are not intrinsically predictive. Therefore, the Adjusted R2 was devised as an amended version of R2 that has been adjusted for the number of predictors in the model.

Mean Squared Error (MSE):

MSE is a common measure to evaluate the accuracy of regression models. It estimates the mean squared error between the predicted values of the target variable and the observed. MSE is calculated as follows:

Root Mean Squared Error (RMSE):

RMSE is one of the popular techniques of measuring the accuracy of regression analysis. It calculates the average squared difference between the predicted values and the actual values of the dependent variable. RMSE is calculated as follows:

Prediction Latency:

Prediction Latency measures computational speed. Combining models in a stacking procedure is computationally expensive, both in terms of training the base models and during prediction time. Prediction latency should ideally be as quick as possible (Barton and Lennox, 2022).

5.0 Results

5.1 App Features Influencing Adoption

Permutation feature importance of selected best features by stacked model which emerged as the best model, is visualised in Figure 4 below with a summary of F-statistics and p-values, denoting the significance of app features on download, with citations from relevant studies with similar findings provided in Table 3.

Figure 5: Permutation Feature importance of selected best features by stacked model which emerged as the best model.

Table 3: This table displays F-statistics and p-values, denoting the significance of app features on download (proxy for adoption), with citations from relevant studies with similar findings.

The application of polynomial regression hypothesis testing to features derived through permutation importance from stacking ensemble yields compelling insights into the determinants of diabetes mobile app adoption as seen in Table 3 above. A low p-value (typically below 0.05) indicates that the corresponding coefficient is statistically significant, meaning it has a significant impact on adoption.

1. Frequency of App Updates (Days since last update): High in importance but not statistically significant (p = 0.1055), suggesting update frequency does not greatly influence app downloads.
2. Ease of Use (Usability): Demonstrated a significant impact on downloads (p = 0.0095), highlighting the critical role of a user-friendly interface.
3. Developer Reputation: Showed high significance (p = 1.123670e-39), underlining the importance of developer credibility in attracting downloads.
4. Prevention Features: Proved significant (p = 2.059865e-25), suggesting a strong user preference for prevention functionalities.
5. Business Models: 'Free' and 'Freemium' significantly influenced downloads, while 'Razor and Blade' showed borderline significance, indicating varying effectiveness of different business models in the app market.
6. Regional-Factor (Taiwan): The variable representing Taiwan as a country-specific factor demonstrated statistical significance in influencing adoption. This finding underscores the particular importance of this regional aspect in the adoption of diabetes mobile health apps.

5.1.1 Hypothesis Result:

This study rejects the null hypothesis, confirming that features such as usability, Free and Freemium business models, developer reputation, regional factors, and prevention features identified through regression analysis significantly influence user adoption rates in diabetes mobile apps, though not all features show a significant impact.

5.2 Predictive Result from Models:

Figure 5 visualizes a comparative analysis of performance metrics with their prediction as well a scatter plot showcasing the actual vs predicted values.

Figure 6: The top chart illustrates a comparative analysis of model performance metrics with an overlaid normalised latency line. The lower charts provide detailed scatter plots comparing the actual vs. predicted values for each model.

The grouped bar chart in Figure 5 visualises that the stacked model emerged as the top performer with the lowest average RMSE (0.4212) and MAE (0.1209) and the highest adjusted R-squared (0.9586) scores, indicating its superior predictive accuracy in the context of diabetes mobile app adoption. Random Forest and XGBoost showed competitive performance, with XGBoost slightly outperforming Random Forest in higher adjusted R2 score (0.9580) and a lower RMSE score (0.4244). Decision Tree presented similar results, with a marginally lower adjusted R-squared score of 0.9405 compared to Random Forest and XGBoost, but impressively, it had the lowest latency score. The SVR had the worst performance metrics. Despite the computational demand of the stacked model, its latency is still suitable for real-time use.
The accompanying scatter plots show the stacked model predictions closely align with actual values, signifying its superior accuracy. On the contrary, the SVR model, with the highest RMSE (1.3896) and a lower adjusted R-squared (0.5680), exhibits the least accuracy in predicting unseen data among the evaluated models.

5.3 User Experience Journey Pain-point Result:

Figure 6 offers an insightful overview of user dissatisfaction and the prevalence of pain points across different stages of the user experience (UX).

Figure 7: Stacked bar chart of UX pain points by category across four stages with an overlaid line graph showing rising dissatisfaction levels from Discovery to Churn.

Source: Personal Jupyter Notebook

Analysing the visualisation in Figure 6, the Discovery Stage of app usage demonstrates that users face moderate pain points related to interface, functionality, and technical issues, yet show low dissatisfaction, indicating initial tolerance. During onboarding, pain points increase slightly with deeper app engagement, but dissatisfaction decreases marginally. In the engagement stage, both pain points and dissatisfaction drop, reflecting growing user comfort and proficiency. However, in the churn stage, despite fewer pain points, dissatisfaction peaks, suggesting that the nature of issues here critically impacts continued app usage.

In the Discovery Stage, users are initially exploring the app. The chart shows a moderate count of pain points, predominantly related to user interface and functionality, as well as technical issues. Despite these initial hurdles, the level of dissatisfaction remains low, suggesting that users might be more forgiving or expect some challenges when they are first introduced to an app.

Moving to the onboarding stage, there's a slight increase in the number of pain points, reflecting the complexities and challenges users face as they begin to engage more deeply with the app's features and settings. Interestingly, the dissatisfaction level decreases marginally compared to the Discovery stage. This could indicate that users anticipate a learning curve during this phase and may appreciate the app's efforts to guide them through it, such as through comprehensive tutorials or responsive customer support.
In the engagement stage, the pain point count decreases significantly, which is indicative of users becoming more comfortable and proficient with the app. Correspondingly, the level of dissatisfaction is low and stable. Users who have continued to this stage likely find the app's offerings satisfactory or have had their issues resolved efficiently, leading to a more seamless experience.

The chart becomes particularly evident in the churn stage. Here, we observe the lowest count of pain points yet the highest level of dissatisfaction. This stark contrast could be attributed to a variety of factors. It may be that the nature of the pain points in this stage are particularly critical issues that directly influence the decision to continue using the app or not. Alternatively, this could also reflect a user base that has dwindled down to only those with significant grievances. The spike in dissatisfaction indicates that the issues at this stage, although fewer in number, are substantial enough to drive users away.

The high dissatisfaction during the Churn stage, despite the lower number of reported pain points, suggests a qualitative difference in issues that users face. This observation allows us to reject the Null Hypothesis (H0), which asserts that there is no significant difference in dissatisfaction levels across UX stages.

6.0 Discussion:

RQ1: What specific features of diabetes mobile apps significantly influence the adoption of diabetes mobile health apps (measured by downloads)?
The research question aimed to identify which specific features of diabetes mobile apps significantly influence their adoption, with a focus on downloads as a proxy for adoption. The findings indicate that several features have a notable impact on user adoption. The significant

influence of developer reputation underscores trust and credibility's role as a significant determinant influencing adoption by users. This aspect, indicative of social influence as discussed by Zhang et al. (2019), suggests users prefer apps from developers with a solid reputation, emphasising the impact of developer credibility on user adoption of the app. Usability as one of the significant features resonates with the findings of Alaslawi et al. (2022) and Krishnan and Selvam (2019). This supports the notion that user-friendly interfaces and practical features like recipe suggestions enhance app downloads. This is also in line with Oughton (2022) emphasis on UI/UX and is echoed by Alaslawi et al. (2022) and Kelly et al. (2018), who also recognised ease of use as crucial for adoption of diabetes mobile apps. Contrary to Krishnan and Selvam's (2019) findings, this study suggests that update frequency does not significantly impact app downloads. This indicates users may prioritise app functionality or content over the frequency of updates. Consistent with the findings of Hou et al. (2018) and Jeffrey et al. (2019), this study also identifies cost as a significant factor influencing app downloads. This indicates a user preference for free, cost-effective apps. The significance of disease management features, such as blood sugar tracking, aligns with Mehraeen et al. (2021). Contrarily, the nutrition feature lacks significance in the study, contrasted with the findings by Alaslawi et al. (2022) and Humble et al. (2016), who noted nutrition as a desired feature.
RQ2: How does dissatisfaction vary across distinct user experience (UX) stages, and what are the characteristic pain points influencing dissatisfaction within each stage?
The research question seeks to understand the variation in user dissatisfaction across different user experience (UX) stages and to identify the specific pain points that contribute to dissatisfaction within each stage for diabetes mobile apps. Unlike previous related studies explored in the literature review, this study adopts a unique approach in examining the user journey through the UX stages of diabetes mobile apps. The findings of this study reveal an interesting and novel pattern of dissatisfaction that correlates with the user journey through the diabetes mobile app user experience (UX) stages.
The initial phases of app interaction, particularly the discovery and onboarding stages, suggest that users often face challenges in initial app usage. The moderate pain points in these stages reflect the learning curve and adjustment period users undergo, in line with the observation that users are initially exploring and understanding the app capabilities. The low dissatisfaction in the engagement stage aligns with the idea that continuous app usage can lead to better diabetes self-management, as users become more comfortable with the app features. This corresponds with findings that effective diabetes apps can enhance user control over their condition (Husted et al., 2018; Garg et al., 2017). The high dissatisfaction despite the low count of pain points in the churn stage is indicative of critical issues not being addressed, which aligns with the related works' emphasis on the importance of usability (Alaslawi et al., 2022) and app functionality (Krishnan & Selvam, 2019). This suggests that even a few unresolved or significant issues can lead to user disengagement and affect adoption negatively.

The need for diabetes apps to continuously capture user attention and stimulate engagement is reflected in the variations of dissatisfaction across different UX stages. Addressing the specific pain points identified in each stage can significantly improve user engagement.
The importance of shared decision-making in app development is underscored by our findings. Involving users in the development process can help identify and mitigate pain points more effectively, leading to apps that better align with user needs and preferences, as suggested by previous studies [Adu et al., 2023; Mehraeen et al., 2023].
Oughton (2022) underscores the critical role of UI/UX design across all industries, including healthcare, emphasising its significance in digital applications. This aspect is particularly crucial in diabetes management, where understanding and addressing high-level user goals and objectives is key. The pivotal question that arises is whether success metrics are uniform for all users within the diabetes spectrum or if they vary based on individual user needs.
Adding to this discourse, Henkel, Randazza-Pade, and Healy (2020) highlight the importance of thoughtfully designing UXs and UIs to enhance health outcomes for people with diabetes. This perspective aligns with our study's findings from the user experience journey pain point analysis in diabetes mobile apps. Our research reveals that the variation of pain points across different UX stages significantly influences user satisfaction and engagement, resonating with the need for a tailored approach in UI/UX design.
RQ3: What are the comparative predictive efficiencies and relevancies of different regression models in determining the factors that influence the adoption of diabetes mobile apps?
The research question investigates the comparative predictive efficiencies and relevancies of different regression models in determining the factors influencing the adoption and user satisfaction of diabetes mobile apps. The findings from the analysis of various regression models reveal noticeable differences in their predictive capabilities, efficiency, and relevance.
In the analysis, an attempt was made to employ Ordinary Least Squares (OLS) regression, following the approach used by Krishnan and Selvam (2019) in their study with the same dataset as used in this study. However, the OLS model showed signs of heteroscedasticity, as indicated by the spread of residuals. With an R-squared value of 0.255, the model does explain some (though minimal) variance in the target variable (downloads). The heteroscedastic nature of residuals suggests that OLS is not the most suitable method for the analysis. Machine learning models suitable for non-linearity were employed, and Random Forest, XGBoost, Decision Tree, and Stacked Model display higher accuracy in prediction. In contrast, models like KNN and SVR were less accurate. The great performance of the ensembles in their predictive capacities, especially stacked ensembles, aligns with the study by Barton and Lennox (2022). Efficiency, as indicated by prediction latency, also varies considerably across models.
An ensemble reduces the risk of choosing a poor individual model, thus improving the model selection procedure (Dietterich, 2000). The stacked ensemble was shown to improve predictive performance, just like in the study by (Barton and Lennox, 2022). Contrary to Krishnan and Selvam (2019) predictive power, the result of this study model demonstrates higher explanatory power.

In conclusion, the adoption of advanced ensemble techniques resulted in models with higher explanatory power than that reported by Krishnan and Selvam (2019). The models used in this study did not only demonstrate enhanced predictive accuracy but also a deeper comprehension of the complex factors influencing the adoption of diabetes mobile apps.

7.0 Recommendations:

In the pursuit of enhancing diabetes mobile health apps, our research meticulously analysed user reviews to derive insights directly reflective of user needs and preferences. This approach ensured that our recommendations are not only data-driven but also resonate with the actual experiences and expectations of end-users.
In enhancing diabetes mobile app adoption, the study points to several key areas for improvement:

1. Technical Glitches and Usability: Apps should focus on stability, with regular and workable updates for bug fixes and performance enhancements for better usability.
2. Connectivity and Integration: It is important to enhance integration with popular health devices for a more complete health management experience.
3. Educational and Supportive Content: Including diverse educational materials aids users in diabetes management. Developing features for various health conditions can also widen the app's appeal.
4. User-Centric Features and Design: Enhancing customisation and accessibility will make apps more user-friendly. Addressing ad experience could improve user experience while maintaining revenue streams.

8.0 Limitation:

The research endeavours faced a significant limitation rooted in the utilisation of the stacked ensemble model. While this approach demonstrated notable improvements in predictive accuracy, it concurrently exhibited a pronounced drawback characterised by heightened computational demands, ultimately manifesting as prolonged prediction latency. This limitation poses a substantial challenge in the context of real-time applications, where swift and responsive predictions are paramount for delivering timely health insights to users.
The elevated computational demand of the stacked ensemble model necessitates a critical examination of its feasibility in scenarios requiring rapid response times. The extended prediction latency could potentially impede the seamless integration of the diabetes mobile health app into users' daily routines, hindering the app's effectiveness in providing timely recommendations and support for managing diabetes.
Moreover, the study's primary focus on key aspects such as usability, developer reputation, and cost-effectiveness, while undoubtedly critical, introduces a potential limitation. By centering attention on these facets, there exists the possibility of overlooking other nuanced factors that wield influence over app adoption dynamics within the realm of diabetes mobile health apps.

9.0 Future Work:

Further research would entail a more thorough assessment of the user experience elements in mobile health apps for diabetes. This entails investigating topics including community involvement, personalisation, privacy issues, and social influence.
In addition, future research should examine state-of-the-art machine learning methods to enhance the personalisation of diabetic mobile health apps by taking into account user preferences, medical history, and lifestyle variables. The efficacy of various personalisation techniques would also be evaluated via A/B testing techniques, with algorithms being continuously improved in response to user input.
Additionally, more study should be done to improve the stacking ensemble's prediction pipeline in order to lower the computing cost of stacking and improve its suitability for real-time applications.
To minimise the size of individual models inside the stacking ensemble, various strategies such as quantisation and knowledge distillation would be investigated for model compression and pruning.
Investigate model pruning techniques to get rid of unnecessary and insignificant parameters, which will lower the total computational expense.
Furthermore, customising app content and recommendations based on unique user profiles and preferences through the use of machine learning algorithms can greatly increase user engagement.
Additionally, implementing mechanisms to ensure that apps adapt to the changing needs and expectations of users by putting in place systems for ongoing user satisfaction surveys, app performance monitoring, and feedback gathering. Additionally, looking into ways to seamlessly integrate electronic health records (EHRs) and technologies used by healthcare providers can improve the overall treatment of diabetes. integrating strategies and ideas from behavioural research into the app to motivate and support users in adhering to diabetes management goals.
Finally, creating applications that work with a variety of online browsers and mobile operating systems helps increase the accessibility and reach of diabetes management solutions.

10.0 Conclusion:

In summary, this study identifies key determinants for the adoption and user satisfaction of diabetes mobile health apps, including developer reputation, usability, cost-effectiveness, etc. It emphasises the necessity of customising app features to meet specific user needs and demographics. The high predictive capacity of the stacking ensemble would be leveraged to analyse user-specific health data and generate personalised treatment plans. This could include dietary recommendations, medication adjustments, and activity suggestions tailored to the user's health profile. Also, it would be leveraged for adaptive notification to determine the most opportune times for delivering notifications based on the user's daily routine and engagement patterns. This ensures that notifications are received and acknowledged when they are most likely to be effective. Additionally, the analysis of user experience stages revealed the importance of addressing distinct user pain points at different stages of interaction with the app. By understanding and mitigating these pain points, app developers can significantly improve the design, functionality, and overall user experience of diabetes mobile health apps. Additionally, the comparison of regression models underscores the importance of selecting the right model for specific analytical objectives, with stacked ensemble models showing improved predictive performance, although it was constrained by high prediction latency. As discussed in the future work, model compression and pruning would be explored to reduce computational cost so as to

give room for real-time predictions. Furthermore, it is imperative to note that the findings of this study were thoroughly validated and justified through hypothesis testing.

References:

Adu, M. D. et al. (2018) “Users’ preferences and design recommendations to promote engagements with mobile apps for diabetes self-management: Multi-national perspectives,” PloS one, 13(12), p. e0208942. doi: 10.1371/journal.pone.0208942.
Alaslawi, H. et al. (2022) “Diabetes self-management apps: Systematic review of adoption determinants and future research agenda,” JMIR diabetes, 7(3), p. e28153. doi: 10.2196/28153.
Buss, V. H. et al. (2022) “A mobile app for prevention of cardiovascular disease and type 2 diabetes mellitus: Development and usability study,” JMIR human factors, 9(2), p. e35065. doi: 10.2196/35065.
Barton, M. & Lennox, B. 2022, "Model stacking to improve prediction and variable importance robustness for soft sensor development", Digital Chemical Engineering, vol. 3, pp. 100034.
Breiman, L. 2001, "Random forests", Machine Learning, vol. 45, pp. 5-32.
Chen, T. and Guestrin, C. (2016) “Xgboost: A scalable tree boosting system,” in Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining, pp. 785–794.
Dietterich, T. G. (2000) “Ensemble methods in machine learning,” in International workshop on multiple classifier systems. Berlin, Heidelberg; Berlin Heidelberg: Springer,
pp. 1–15.
Eng, D. S. and Lee, J. M. (2013) “The Promise and Peril of Mobile Health Applications for Diabetes and Endocrinology: Mobile health applications in diabetes and endocrinology,” Pediatric diabetes, 14(4), pp. 231–238. doi: 10.1111/pedi.12034.
Friedman, J.H., 2017. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. New York: Springer.
Garg, S. K., Shah, V. N., Akturk, H. K., Beatson, C., & Snell-Bergeon, J. K. (2017). Role of Mobile Technology to Improve Diabetes Care in Adults with Type 1 Diabetes: The Remote-T1D Study iBGStar in Type 1 Diabetes Management. Diabetes Ther, 8(4), 811-819. doi:10.1007/s13300- 017-0272-5
Husted, G. R., Weis, J., Teilmann, G., & Castensøe-Seidenfaden, P. (2018). Exploring the influence of a smartphone app (Young with Diabetes) on young people's self-management: Qualitative study. JMIR Mhealth Uhealth, 6(2), e43. doi:10.2196/mhealth.8876.
Haoues, M., Mokni, R. and Sellami, A. (2023) “Machine learning for mHealth apps quality evaluation: An approach based on user feedback analysis,” Software quality journal, 31(4), pp. 1179–1209. doi: 10.1007/s11219-023-09630-8.
Hou, C., Xu, Q., Diao, S., Hewitt, J., Li, J., & Carter, B. (2018). Mobile phone applications and self- management of diabetes: A systematic review with meta- analysis, meta-regression of 21 randomizedtrials and GRADE. Diabetes Obes Metab, 20(8), 2009-2013. doi:10.1016/j.diabet.2018.03.010.
Humble, J. R. et al. (2016) “Use of and interest in mobile health for diabetes self-care in vulnerable populations,” Journal of telemedicine and telecare, 22(1), pp. 32–38. doi: 10.1177/1357633X15586641.
IDF Diabetes Atlas, Eight 8th Edn Brussels: International Diabetes Federation (2017).
Jeffrey, B. et al. (2019) “Mobile phone applications and their use in the self-management of Type 2 Diabetes Mellitus: a qualitative study among app users and non-app users,” Diabetology & metabolic syndrome, 11(1). doi: 10.1186/s13098-019-0480-4.
Kelly, L., Jenkinson, C. and Morley, D. (2018) “Experiences of using web-based and mobile technologies to support self-management of type 2 diabetes: Qualitative study,” JMIR diabetes, 3(2), p. e9. doi: 10.2196/diabetes.9743.
Krishnan, G. and Selvam, G. (2019) 'Factors influencing the download of Mobile Health Apps: Content Review-led regression analysis', Health Policy and Technology, 8(4), pp. 356–364. doi:10.1016/j.hlpt.2019.09.001.
Krishnan, G. (2020) Data for: Factors influencing the download of Mobile Health Apps: Content Review cum regression analysis, Mendeley Data. Available at: https://data.mendeley.com/datasets/w396pxs7k7/1 [Downloaded 03 September 2023].
Lam, L. W. and Harrison-Walker, L. J. (2003) “Toward an objective-based typology of e- business models,” Business horizons, 46(6), pp. 17–26. doi: 10.1016/s0007- 6813(03)00084-3.
Mehraeen, E. et al. (2021) “Identifying features of a mobile-based application for self- care of people living with T2DM,” Diabetes research and clinical practice, 171(108544),
p. 108544. doi: 10.1016/j.diabres.2020.108544.
Oughton, A. 2022, "Chapter 2 - Building digital health tools for diabetes: how user experience research and user interface design can improve digital health adoption" in Diabetes Digital Health and Telehealth, eds. D.C. Klonoff, D. Kerr & E.R. Weitzman, Academic Press, pp. 15-27.
Sagi, O. (2018) “Rokach Ensemble learning: a survey Wiley Interdisciplinary Reviews,” Data Mining and Knowledge Discovery, 8(4), p. 1249.
Spiess, A. N. and Neumeyer, N. (2010) “An evaluation of R2 as an inadequate measure for nonlinear models in pharmacological and biochemical research: a Monte Carlo approach,” BMC Pharmacology, 10(6).
Wang, F. et al. (2020) “Fang A novel method with stacking learning of data-driven soft sensors for mud concentration in a cutter suction dredger Sensors,” 20(21), p. 6075.
Williams, J. P. and Schroeder, D. (2015) “Popular glucose tracking apps and use of mHealth by Latinos with diabetes: Review,” JMIR mHealth and uHealth, 3(3), p. e84. doi: 10.2196/mhealth.3986.
Zhang, Y. et al. (2019) “Factors influencing patients’ intentions to use diabetes management apps based on an extended Unified Theory of Acceptance and use of technology model: Web-based survey,” Journal of medical internet research, 21(8), p. e15023. doi: 10.2196/15023.
Trawley, S., Baptista, S., Browne, J.L., Pouwer, F. and Speight, J. (2017). The Use of Mobile Applications Among Adults with Type 1 and Type 2 Diabetes: Results from the Second MILES—Australia (MILES-2) Study. Diabetes Technology & Therapeutics,
19(12), pp.730–738. doi:https://doi.org/10.1089/dia.2017.0235.
Ernsting C, Stühmann LM, Dombrowski SU, Voigt-Antons J, Kuhlmey A, Gellert P. Associations of health app use and perceived effectiveness in people with cardiovascular diseases and diabetes: population-based survey. JMIR Mhealth Uhealth 2019 Mar 28;7(3)

.
Jeffrey, B., Bagala, M., Creighton, A., Leavey, T., Nicholls, S., Wood, C., Longman, J., Barker, J. and Pit, S. (2019). Mobile Phone Applications and Their Use in the self- management of Type 2 Diabetes Mellitus: a Qualitative Study among App Users and non-app Users. Diabetology & Metabolic Syndrome, [online] 11(1), pp.1–17. doi:https://doi.org/10.1186/s13098-019-0480-4.
Peng, W., Yuan, S. and Holtz, B.E. (2016). Exploring the Challenges and Opportunities of Health Mobile Apps for Individuals with Type 2 Diabetes Living in Rural Communities. Telemedicine and e-Health, 22(9), pp.733–738. doi:https://doi.org/10.1089/tmj.2015.0180.
Surkan, P.J., Mezzanotte, K.S., Sena, L.M., Chang, L.W., Gittelsohn, J., Trolle Lagerros, Y., Quinn, C.C. and Zachary, W.W. (2019). Community-Driven Priorities in Smartphone Application Development: Leveraging Social Networks to Self-Manage Type 2 Diabetes in a Low-Income African American Neighborhood. International Journal of Environmental Research and Public Health, 16(15), p.2715. doi:https://doi.org/10.3390/ijerph16152715.
Thies, K., Anderson, D. and Cramer, B. (2017). Lack of Adoption of a Mobile App to Support Patient Self-Management of Diabetes and Hypertension in a Federally Qualified Health Center: Interview Analysis of Staff and Patients in a Failed Randomized
Trial. JMIR Human Factors, 4(4), p.e24. doi:https://doi.org/10.2196/humanfactors.7709.
Scheibe, M., Reichelt, J., Bellmann, M. and Kirch, W. (2015). Acceptance Factors of Mobile Apps for Diabetes by Patients Aged 50 or Older: A Qualitative Study. Medicine 2.0, [online] 4(1), p.e1. doi:https://doi.org/10.2196/med20.3912.
Torbjørnsen, A., Ribu, L., Rønnevig, M., Grøttland, A. and Helseth, S. (2019). Users’ acceptability of a mobile application for persons with type 2 diabetes: a qualitative study. BMC Health Services Research, 19(1). doi:https://doi.org/10.1186/s12913-019- 4486-2.
Tanenbaum, M.L., Bhatt, H.B., Thomas, V.A. and Wing, R.R. (2016). Use of self- monitoring tools in a clinic sample of adults with type 2 diabetes. Translational Behavioral Medicine, [online] 7(2), pp.358–363. doi:https://doi.org/10.1007/s13142-016- 0418-4.
Pludwinski, S., Ahmad, F., Wayne, N. and Ritvo, P. (2015). Participant experiences in a smartphone-based health coaching intervention for type 2 diabetes: A qualitative inquiry. Journal of Telemedicine and Telecare, 22(3), pp.172–178. doi:https://doi.org/10.1177/1357633x15595178.
Brandt, L.R., Hidalgo, L., Diez-Canseco, F., Araya, R., Mohr, D.C., Menezes, P.R. and Miranda, J.J. (2019). Addressing Depression Comorbid With Diabetes or Hypertension in Resource-Poor Settings: A Qualitative Study About User Perception of a Nurse- Supported Smartphone App in Peru. JMIR Mental Health, 6(6), p.e11701. doi:https://doi.org/10.2196/11701.
Kabeza, C.B., Harst, L., Schwarz, P.E.H. and Timpel, P. (2019). Assessment of Rwandan diabetic patients’ needs and expectations to develop their first diabetes self-management smartphone application (Kir’App). Therapeutic Advances in Endocrinology and Metabolism, 10, p.204201881984531. doi:https://doi.org/10.1177/2042018819845318.
Kayyali, R., Peletidi, A., Ismail, M., Hashim, Z., Bandeira, P. and Bonnah, J. (2017). Awareness and Use of mHealth Apps: A Study from England. Pharmacy, [online] 5(2),
p.33. doi:https://doi.org/10.3390/pharmacy5020033.