1. Please check and comment entries here.
Table of Contents

    Topic review

    Digital Health Intervention Engagement

    Subjects: Others
    View times: 14
    Submitted by: Tessa Delaney

    Definition

    There has been a proliferation of digital health interventions (DHIs) targeting dietary intake. Despite their potential, the effectiveness of DHIs are thought to be dependent, in part, on user engagement. However, the relationship between engagement and the effectiveness of dietary DHIs is not well understood.

    1. Introduction

    The use of digital health interventions (DHI) has been recommended as a strategy to improve population dietary intake [1]. The World Health Organization refers to ‘digital health’ as the use of digital, mobile, and wireless technologies to support the achievement of health objectives and is both inclusive of m-health and e-health [2]. Digital health technologies may include mobile phones, portable computer tablets (e.g., iPads), web-based interventions, smartphone applications (apps) and wearable devices [3]. With 3.9 billion internet users and the potential to reach over 90% of the global population [4], DHIs, once developed, can be a cost-effective way of delivering interventions to large numbers of individuals and organizations in the population, and can be delivered with high fidelity and at low cost to a wide variety of populations, including disadvantaged groups [5].

    Despite the promise of DHIs, systematic reviews evaluating the effectiveness of smartphone applications [6] and web-based interventions [7] provide mixed evidence on their effectiveness in improving dietary intake, with a lack of user engagement hypothesized as a limiting factor [8][9]. Engagement has recently been defined as both i) the extent of DHI usage, such as the amount, frequency, duration and depth of the DHI accessed, and ii) a subjective experience characterized by attention, interest and affect [10]. Whilst suggested to be important, the association between the characteristics of engagement and health behavior is not well understood [11]. As such, having a greater understanding of the relationship between engagement and dietary intake will likely provide an opportunity to optimize the impact of DHIs.

    A 2011 systematic review of 33 studies examining the effect of engagement with web-based interventions and health outcomes found a positive relationship between DHI usage and fruit-and-vegetable intake, physical activity, weight management, and reductions in smoking and smokeless tobacco use [3]. The review found a positive relationship between DHI usage and improvement in dietary intake. However, the review included a narrow definition of engagement (e.g., focused on usage only), did not include a comprehensive search (e.g., included five keywords in the search strategy), and was restricted to web-based interventions, only, without considering other digital health technologies, such as m-health and smartphone applications. Furthermore, the systematic review identified just one study that assessed the association between DHI usage (logins) and dietary intake [3]. This randomized controlled trial of an online intervention found that more frequent website visits were associated with increased fruit-and-vegetable intake ( p < 0.001) [12]. Since the 2011 review, there have been a large increase in research of DHIs targeting dietary intake [7][13]. This provides an opportunity to better understand the association between DHI engagement and dietary intake.

    2. Research Methods

    Researchers included study designs that quantitatively examined an association between a measure of engagement with a DHI and any measure of dietary intake. Specifically, study designs could have included retrospective, prospective (e.g., randomized controlled trials, cohort studies), cross-sectional, before and after studies and interrupted time series studies. Engagement was defined as both the extent of the usage of the program (e.g., number of logins, time on site and activities completed) as well as the subjective experience, including measures of attention, interest, and affect [10]. DHIs were defined as the use of digital, mobile, and wireless technologies to support the achievement of health objectives. This was inclusive of both m-health and e-health. DHIs included, but were not limited to, portable computer tablets (e.g., iPads), web-based interventions and smartphone applications (apps) [2].

    Pairs of review authors (JD, KO or AB, TD or TD, MM) assessed methodological quality of studies, independently, using the Newcastle—Ottawa Scale for cohort [14] or cross-sectional studies [15]. Researchers defined cross-sectional as those studies using a single time-point of data for the dietary intake measure (e.g., follow-up), whereas cohort studies were those that used multiple time-points of data and calculated change over time (e.g., change from baseline to follow-up). The Newcastle—Ottawa Scale utilizes a star system to assess the methodological quality of cohort and cross-sectional studies. The cohort tool assigns a maximum of nine stars across three domains: (1) selection of study groups (up to four stars); (2) the comparability of these groups (up to two stars); and (3) assessment of outcomes (up to three stars). The cross-sectional tool assigns a maximum of ten stars across the same three domains: (1) selection of study groups (up to five stars), (2) the comparability of these groups (up to two stars), and (3) assessment of outcomes (up to three stars) ( Supplementary File S3 ).

    Pooled quantitative synthesis was not possible due to high heterogeneity across the studies included in the review. An overview of all associations including direction, strength, and favorability, along with the characteristics of the included studies, are summarized, in full, in Table 1 .

    Table 1. Characteristics of included studies.
    Author and Study Characteristics Description of Digital Health Intervention Engagement Outcome/s Dietary Outcome/s Association (b) Direction of Association (c) Favorable (d)
    Author: Alexander 2010; (also reported by Couper 2010)
    Design (a): cohort
    N = 2513 (baseline)
    Age: 46.3 (SD 10.8)
    Female = 69%
    Type: Website
    Description: Three-arm website intervention, all arms included access to a basic website with varying levels of tailoring. Arm 1 was the basic site, Arm 2 was a tailored website, Arm 3 was a tailored website with motivational interviewing via email. Web sessions were delivered at 1, 3, 13, and 15 weeks. Participants received $2 incentive prior to entering study & $20 for completing study
    Intervention target: Adults (21–65 years) with no existing health conditions who were registered in a health care system database
    Total duration of DHI: 12 months *
    Logins dichotomized into high (>14 logins); medium (7–13 logins); and low (<7 logins) groups Change in mean servings of fruit and vegetables from baseline to 12 months using a 16-item valid FFQ low (mean change 2.1); medium (mean change 2.5); high (mean change 3.1) p < 0.001 +
    Breadth-the sum of four measures, standardized by dividing by their standard deviation. including: total session accesses, unique session access, total special feature accesses, total time online in minutes Change in mean servings of fruit and vegetables from baseline to 12 months usinga 16-item valid FFQ Coefficients not presented;
    p < 0.001
    +
    Change in mean servings of fruit and vegetables from baseline to 12 months usinga 2-item valid FFQ Coefficients not presented;
    p < 0.001
    +
    Depth-sum of average total special features sessions; standardized minutes spent online subtracted by twice total number (standardized) of unique sessions Change in mean servings of fruit and vegetables from baseline to 12 months using a 16-item valid FFQ Coefficients not presented;
    p = 0.83
    0 N/A
    Change in mean servings of fruit and vegetables from baseline to 12 months using a 2-item valid FFQ Coefficients not presented;
    p = 0.92
    0 N/A
    Author: Buller 2008; (also reported by Woodall 2007)
    Design (a): cohort
    N = 380 (baseline)
    Age (c): <29 years = 35%
    Female = 88%
    Type: Website
    Description: Fruit and vegetable nutrition education website (password protected), participants were contacted by research team to log onto website once each month, every 2 months participants received ‘small gift’ as a reminder to visit website, routine email notifications were sent announcing new content.
    Intervention target: Adults (>18 years old), English speaking and living in Southwestern USA for at least 6 months.
    Total duration of DHI: 4 months
    Time on website (mean minutes) Change in mean servings of fruit-and-vegetable intake from baseline to 4 months using valid all day screener (ranked pre- and post-test) Unadjusted: R = 0.14, p = 0.004 +
    Adjusted: Estimate = 0.74, SD = 0.19, t(df = 414) = 3.87, p = 0.001 +
    Time on website (mean minutes) Change in mean servings of fruit-and-vegetable intake from baseline to 4 months using single item screener (ranked pre- and post-test) OR (95% CI)
    1.010 (1.003, 1.018) per minute of use
    +
    Time on website features (mean minutes) Change in mean servings of fruit-and-vegetable intake from baseline to 4 months using valid all day screener (ranked pre- and post-test) 17 associations
    Range of means (SD): 0.009 (0.096) to 13.745 (21.203)
    Range of Spearman correlation:
    −0.076 to 0.185
    Range of p value: 0.0064 to 0.9189 (only 3 significant)
    N/A (e) N/A
    Number of logins within 5 days of an email Change in mean servings of fruit-and-vegetable intake from baseline to 4 months using valid FFQ coefficient = 0.14, p = 0.049 +
    Proportion of logins after email Change in mean servings of fruit-and-vegetable intake from baseline to 4 months using valid FFQ coefficient = 0.11, p = 0.12 0 N/A
    Author: Kothe 2014
    Design (a): cohort
    N = 217 (baseline)
    Age: 18.92 (SD 1.37)
    Female = 77.3%
    Type: Email intervention
    Description: Email intervention with two levels of message frequency. Participants in high frequency intervention arm received emails daily (27 emails in total) and those in low frequency arm received emails every 3 days (9 emails in total). Course credit was provided for participating students
    Intervention target: Adults (>18 years) who were an undergraduate psychology student at an Australian University
    Total duration of DHI: 30 days
    Subjective experience using Likert scale:
    Interest
    Change in fruit-and-vegetable intake scores (servings/day) from baseline to 30 days using self-report e.g., “How many servings of fruit did you eat yesterday?” Correlation = 0.163, p < 0.05 +
    - Credibility Correlation = 0.002,
    p = ‘not significant’
    0 N/A
    - Logical Correlation = −0.034, p = ‘not significant’ 0 N/A
    - Easy to understand Correlation = 0.021,
    p = ‘not significant’
    0 N/A
    - Relevant Correlation = 0.102,
    p = ‘not significant’
    0 N/A
    - Useful Correlation = 0.149,
    p < 0.05
    +
    - Complete Correlation = 0.146,
    p < 0.05
    +
    - Too long Correlation = −0.032,
    p = ‘not significant’
    0 N/A
    - Annoying Correlation = −0.104,
    p = ‘not significant’
    0 N/A
    - Too many emails Correlation = −0.078,
    p = ‘not significant’
    0 N/A
    - Confusing Correlation = 0.067,
    p = ‘not significant’
    0 N/A
    Author: Lippke 2016
    Design of association(a): cohort
    N = 701 (at association)
    Age: 38.71
    Female = 84%
    Type: Website
    Description: One-off action-planning and coping-planning website aimed to improve fruit-and-vegetable intake. As an incentive for study participation, individuals were able to take part in an optional raffle in which they could win attractive gift certificates for an online bookstore
    Intervention target: Adults
    Total duration of DHI: 1 month
    Engagement survey score using Likert scale Change in fruit-and-vegetable intake scores from baseline to one month (servings/day) using valid ‘open answer’ questionnaire e.g., “how many servings of (a) fruit…and (b) vegetables…do you eat on average per day? Correlation = 0.01, p = ‘not significant’,non-linear relationship observed 0 N/A
    Author: Moore 2008
    Design (a): cohort
    N = 181 (at association)
    Age: not reported
    Female = 59% #
    Type: Website
    Description: Password-protected website on healthy eating, content was posted each Friday with weekly reminder emails sent to participants. Dietary advice was based on the DASH diet (Dietary Approaches to Stop Hypertension).
    Intervention target: Adult employees of a US based infrastructure company
    Total duration of DHI: 12 months
    Number of logins Change in fruit servings from baseline to 12 months using valid FFQ p = 0.03 +
    Change in vegetable servings from baseline to 12 months using valid FFQ p = ‘not significant’ 0 N/A
    Change in grains servings from baseline to 12 months using valid FFQ p = ‘not significant’ 0 N/A
    Change in dairy servings from baseline to 12 months using valid FFQ p = ‘not significant’ 0 N/A
    Change in meat & fish servings from baseline to 12 months using valid FFQ p = ‘not significant’ 0 N/A
    Change in nut & beans servings from baseline to 12 months using valid FFQ p = ‘not significant’ 0 N/A
    Change in added fats servings from baseline to 12 months using valid FFQ p = ‘not significant’ 0 N/A
    Change in sweets servings from baseline to 12 months using valid FFQ p = ‘not significant’ 0 N/A
    Author: Nour 2019
    Design (a): Cross-sectional
    N = 97 (baseline) Age: 24.8 (SD 3.4)
    Female = 60%
    Type Standard App OR gamified app +/- Facebook
    Description: Standard app of goal setting and self-monitoring with feedback on vegetable intake, Gamified app included rewards as incentivization. Facebook included cooking videos addressing known barriers shared by a dietician daily.
    Intervention target: Adults 18–30 years, who owned a smartphone and lived in New South Wales, Australia
    Total duration of DHI: 4 weeks
    Total days of app engagement via recorded logins in standard app Change in vegetable intake (servings/ day) from baseline to 4 weeks using valid short questionnaires r = 1; n = 23; p < 0.00001 +
    Total days of app engagement via recorded logins in gamified app   r = 0.64; n = 24; p = 0.001 +
    Frequency of recording vegetable intake via app analytics in standard app   r = 0.49; n = 23; p = 0.02 +
    Frequency of recording vegetable intake via app analytics in gamified app   r = 0.35; n = 24; p = 0.09 0 N/A
    Author: Rodgers 2016
    Design (a): cohort
    N = 46 (baseline)
    Age: 18.96 (SD 0.76)
    Female = 100%
    Type: Website + SMS
    Description: Participants were encouraged to take photos of meals using their mobile phone and upload them to a website (Photobucket) and received 3 x motivational text messages/day at mealtimes to encourage healthy eating. Intervention target: Full time female undergraduate college students (>18 years)
    Total duration of DHI: 3 weeks
    Number of photos posted (logins) Vegetable intake (servings/day) using a valid 2-item FFQ Estimate = 0.012, SE = 0.008, p = ‘not significant’ 0 N/A
      Fruit intake (servings/day) using a valid 2-item FFQ Estimate = 0.017, SE = 0.008, p < 0.05 +
      Log of calories from sugar-sweetened beverages using a ‘beverage intake questionnaire’ Estimate = 0.007, SE = 0.009, p = ‘not significant’ 0 N/A

    For the vote count, studies were counted once for each engagement construct where they provided one or more measures of association. For example, if a study reported two associations of engagement (e.g., time on-site and logins) both would have been included in the vote count synthesis. If there were multiple tests of association reported using the same engagement measure and same dietary outcome in the one study (e.g., multiple associations reported for ‘time on site’ and fruit and vegetable intake), we used the following inclusion criteria to select the association of interest from each study for inclusion in the vote count: If a study had multiple associations using the same engagement measure and same dietary outcome, preference was given to the dietary outcome assessed using the instrument judged by the authors (in the absence of published reliability or validity data) to be most comprehensive. For example, if a study reported two associations including i) ‘time on website’ using an ‘all day’ fruit-and-vegetable screener and ii) ‘time on website’ using a ‘single-item’ fruit-and-vegetable screener, preference was given to the ‘all day screener’ as it is the more comprehensive outcome measure for fruit-and-vegetable intake. If multiple models were presented assessing the association between the same dietary outcome and same engagement measure (e.g., unadjusted and adjusted) we gave preference to the adjusted model. If multiple engagement measures were used and they all assessed the same type of engagement outcome (e.g., time on site) we selected the most complete and inclusive. For example, ‘total time on website’ was given preference to ‘time on a specific website feature’.

    3. Key Strength

    A key strength of this review was the comprehensive search strategy, which included screening of 10,653 citations, utilizing published search filters and manual searching in relevant journals and of grey literature. Another strength was that it included measures of subjective experience, a key engagement outcome often overlooked in previous research and suggested to be an important predictor of DHI effectiveness [10]. Despite this, the review should be interpreted in the context of its limitations. First, the heterogeneity of engagement outcomes in studies precluded meta-analyses, an issue reported in previous reviews of engagement [3][21][22]. As such researchers provide a narrative synthesis and rely on methods such as vote counting in order to synthesize study findings. Second, the inclusion of additional databases in the search may have resulted in additional included studies. Finally, all but two studies were rated as ‘poor quality’, studies were primarily downgraded due to their analysis not being described clearly, which was often a result of the association being included as exploratory or process data rather than a primary or secondary study outcome.

    4. Conclusions

    Current reviews address an important knowledge gap in the engagement literature and is the first to synthesize the association between DHI engagement and dietary intake. The findings suggest there is some evidence supporting an association with usage, however this was inconsistent. No evidence was found regarding an association with subjective experience. Whilst it has been hypothesized that the modest effects of dietary DHIs are due to poor engagement [13], the findings do not yet support this and provide little guidance as to which components of engagement to target to enhance the effectiveness of DHIs. Given the reliance on many public health nutrition strategies on DHIs [23][24], a better understanding of the nature of the relationship is a priority for the field. Specifically, the development and application of consistent and comprehensive measures of the multiple dimensions of engagement is required, and the use of more nuanced, mixed-method, and qualitative approaches may be required to better understand the relationship between DHIs and engagement. In particular, it has been hypothesized that the relationship between engagement and behavior change is unlikely to be linear [25] and that greater engagement may not necessarily yield greater effects on behavior change.

    This entry is adapted from 10.3390/nu13093281

    References

    1. World Health Organization. Digital Health. 2019. Available online: https://www.who.int/behealthy/digital-health (accessed on 10 April 2019).
    2. World Health Organization. Monitoring and Evaluating Digital Health Interventions: A Practical Guide to Conducting Research and Assessment; WHO: Geneva, Switzerland, 2016.
    3. Donkin, L.; Christensen, H.; Naismith, S.; Neal, B.; Hickie, I.B.; Glozier, N. A Systematic Review of the Impact of Adherence on the Effectiveness of e-Therapies. J. Med. Internet Res. 2011, 13, e52.
    4. International Telecommunications Union. Statistics. 2018. Available online: https://www.itu.int/en/ITU-D/Statistics/Pages/stat/default.aspx (accessed on 20 August 2019).
    5. Bastawrous, A.; Armstrong, M. Mobile health use in low- and high-income countries: An overview of the peer-reviewed literature. J. R. Soc. Med. 2013, 106, 130–142.
    6. Schoeppe, S.; Alley, S.; Van Lippevelde, W.; Bray, N.A.; Williams, S.L.; Duncan, M.J.; Vandelanotte, C. Efficacy of interventions that use apps to improve diet, physical activity and sedentary behaviour: A systematic review. Int. J. Behav. Nutr. Phys. Act. 2016, 13, 127.
    7. Kohl, L.F.M.; Crutzen, R.; De Vries, N.K. Online Prevention Aimed at Lifestyle Behaviors: A Systematic Review of Reviews. J. Med. Internet Res. 2013, 15, e146.
    8. Grady, A.; Yoong, S.L.; Sutherland, R.; Lee, H.; Nathan, N.; Wolfenden, L. Improving the public health impact of eHealth and mHealth interventions. Aust. N. Z. J. Public Health 2018, 42, 118–119.
    9. Cole-Lewis, H.; Ezeanochie, N.; Turgiss, J. Understanding Health Behavior Technology Engagement: Pathway to Measuring Digital Behavior Change Interventions. JMIR Form. Res. 2019, 3, e14052.
    10. Perski, O.; Blandford, A.; West, R.; Michie, S. Conceptualising engagement with digital behaviour change interventions: A systematic review using principles from critical interpretive synthesis. Transl. Behav. Med. 2017, 7, 254–267.
    11. Pham, Q.; Graham, G.; Carrion, C.; Morita, P.P.; Seto, E.; Stinson, J.N.; A Cafazzo, J. A Library of Analytic Indicators to Evaluate Effective Engagement with Consumer mHealth Apps for Chronic Conditions: Scoping Review. JMIR mHealth uHealth 2019, 7, e11941.
    12. Alexander, G.L.; McClure, J.B.; Calvi, J.H.; Divine, G.W.; Stopponi, M.A.; Rolnick, S.J.; Heimendinger, J.; Tolsma, D.; Resnicow, K.; Campbell, M.K.; et al. A Randomized Clinical Trial Evaluating Online Interventions to Improve Fruit and Vegetable Consumption. Am. J. Public Health 2010, 100, 319–326.
    13. Young, C.; Campolonghi, S.; Ponsonby, S.; Dawson, S.L.; O’Neil, A.; Kay-Lambkin, F.; McNaughton, S.; Berk, M.; Jacka, F.N. Supporting Engagement, Adherence, and Behavior Change in Online Dietary Interventions. J. Nutr. Educ. Behav. 2019, 51, 719–739.
    14. Wells, G.; Shea, B.; O’Connell, D.; Peterson, J.; Welch, V.; Losos, M.; Tugwell, P. Newcastle-Ottawa Quality Assessment Scale Cohort Studies; University of Ottawa: Ottawa, ON, Canada, 2014.
    15. Modesti, P.A.; Reboldi, G.; Cappuccio, F.P.; Agyemang, C.; Remuzzi, G.; Rapi, S.; Perruolo, E.; Parati, G.; ESH Working Group on CV Risk in Low Resource Settings. Panethnic Differences in Blood Pressure in Europe: A Systematic Review and Meta-Analysis. PLoS ONE 2016, 11, e0147601.
    16. Yoong, S.L.; Grady, A.; Wiggers, J.H.; Stacey, F.G.; Rissel, C.; Flood, V.; Finch, M.; Wyse, R.; Sutherland, R.; Salajan, D.; et al. Child-level evaluation of a web-based intervention to improve dietary guideline implementation in childcare centers: A cluster-randomized controlled trial. Am. J. Clin. Nutr. 2020, 111, 854–863.
    17. Moore, T.J.; Alsabeeh, N.; Apovian, C.; Murphy, M.C.; Coffman, G.; Cullum-Dugan, D.; Jenkins, M.; Cabral, H. Weight, Blood Pressure, and Dietary Benefits After 12 Months of a Web-based Nutrition Education Program (DASH for Health): Longitudinal Observational Study. J. Med. Internet Res. 2008, 10, e52.
    18. Rodgers, R.F.; Pernal, W.; Matsumoto, A.; Shiyko, M.; Intille, S.; Franko, D.L. Capitalizing on mobile technology to support healthy eating in ethnic minority college students. J. Am. Coll. Health 2016, 64, 125–132.
    19. Yardley, L.; Spring, B.J.; Riper, H.; Morrison, L.G.; Crane, D.H.; Curtis, K.; Merchant, G.C.; Naughton, F.; Blandford, A. Understanding and Promoting Effective Engagement With Digital Behavior Change Interventions. Am. J. Prev. Med. 2016, 51, 833–842.
    20. Perski, O.; Blandford, A.; Garnett, C.; Crane, D.; West, R.; Michie, S. A self-report measure of engagement with digital behavior change interventions (DBCIs): Development and psychometric evaluation of the “DBCI Engagement Scale”. Transl. Behav. Med. 2020, 10, 267–277.
    21. Milward, J.; Drummond, C.; Fincham-Campbell, S.; DeLuca, P. What makes online substance-use interventions engaging? A systematic review and narrative synthesis. Digit. Health 2018, 4.
    22. Rhodes, A.; Smith, A.D.; Chadwick, P.; Croker, H.; Llewellyn, C.H. Exclusively Digital Health Interventions Targeting Diet, Physical Activity, and Weight Gain in Pregnant Women: Systematic Review and Meta-Analysis. JMIR mHealth uHealth 2020, 8, e18255.
    23. Livingstone, K.M.; Celis-Morales, C.; Navas-Carretero, S.; San-Cristobal, R.; Macready, A.L.; Fallaize, R.; Forster, H.; Woolhead, C.; O’Donovan, C.B.; Marsaux, C.F.; et al. Effect of an Internet-based, personalized nutrition randomized trial on dietary changes associated with the Mediterranean diet: The Food4Me Study. Am. J. Clin. Nutr. 2016, 104, 288–297.
    24. Garcia-Ortiz, L.; Recio-Rodríguez, J.I.; Agudo-Conde, C.; Patino-Alonso, M.C.; Maderuelo-Fernandez, J.-A.; Gento, I.R.; Puig, E.P.; González-Viejo, N.; Arietaleanizbeaskoa, M.S.; Schmolling-Guinovart, Y.; et al. Long-Term Effectiveness of a Smartphone App for Improving Healthy Lifestyles in General Population in Primary Care: Randomized Controlled Trial (Evident II Study). JMIR mHealth uHealth 2018, 6, e107.
    25. Lippke, S.; Corbet, J.M.; Lange, D.; Parschau, L.; Schwarzer, R. Intervention Engagement Moderates the Dose–Response Relationships in a Dietary Intervention. Dose-Response 2016, 14.
    More