1. Introduction

The use of digital health interventions (DHI) has been recommended as a strategy to improve population dietary intake ^[1][6]. The World Health Organization refers to ‘digital health’ as the use of digital, mobile, and wireless technologies to support the achievement of health objectives and is both inclusive of m-health and e-health ^[2][7]. Digital health technologies may include mobile phones, portable computer tablets (e.g., iPads), web-based interventions, smartphone applications (apps) and wearable devices ^[3][8]. With 3.9 billion internet users and the potential to reach over 90% of the global population ^[4][9], DHIs, once developed, can be a cost-effective way of delivering interventions to large numbers of individuals and organizations in the population, and can be delivered with high fidelity and at low cost to a wide variety of populations, including disadvantaged groups ^[5][10].

Despite the promise of DHIs, systematic reviews evaluating the effectiveness of smartphone applications ^[6][11] and web-based interventions ^[7][12] provide mixed evidence on their effectiveness in improving dietary intake, with a lack of user engagement hypothesized as a limiting factor ^[8][9][13,14]. Engagement has recently been defined as both i) the extent of DHI usage, such as the amount, frequency, duration and depth of the DHI accessed, and ii) a subjective experience characterized by attention, interest and affect ^[10][15]. Whilst suggested to be important, the association between the characteristics of engagement and health behavior is not well understood ^[11][16]. As such, having a greater understanding of the relationship between engagement and dietary intake will likely provide an opportunity to optimize the impact of DHIs.

A 2011 systematic review of 33 studies examining the effect of engagement with web-based interventions and health outcomes found a positive relationship between DHI usage and fruit-and-vegetable intake, physical activity, weight management, and reductions in smoking and smokeless tobacco use ^[3][8]. The review found a positive relationship between DHI usage and improvement in dietary intake. However, the review included a narrow definition of engagement (e.g., focused on usage only), did not include a comprehensive search (e.g., included five keywords in the search strategy), and was restricted to web-based interventions, only, without considering other digital health technologies, such as m-health and smartphone applications. Furthermore, the systematic review identified just one study that assessed the association between DHI usage (logins) and dietary intake ^[3][8]. This randomized controlled trial of an online intervention found that more frequent website visits were associated with increased fruit-and-vegetable intake ( p < 0.001) ^[12][17]. Since the 2011 review, there have been a large increase in research of DHIs targeting dietary intake ^[7][13][12,18]. This provides an opportunity to better understand the association between DHI engagement and dietary intake.

Therefore, the aim of the review is to systematically review the literature to describe the association between objective DHI engagement (both usage and subjective experience) and dietary intake.

2. RMatesearchrials and Methods

ResWearchers included study designs that quantitatively examined an association between a measure of engagement with a DHI and any measure of dietary intake. Specifically, study designs could have included retrospective, prospective (e.g., randomized controlled trials, cohort studies), cross-sectional, before and after studies and interrupted time series studies. Engagement was defined as both the extent of the usage of the program (e.g., number of logins, time on site and activities completed) as well as the subjective experience, including measures of attention, interest, and affect ^[10][15]. DHIs were defined as the use of digital, mobile, and wireless technologies to support the achievement of health objectives. This was inclusive of both m-health and e-health. DHIs included, but were not limited to, portable computer tablets (e.g., iPads), web-based interventions and smartphone applications (apps) ^[2][7].

Pairs of review authors (JD, KO or AB, TD or TD, MM) assessed methodological quality of studies, independently, using the Newcastle—Ottawa Scale for cohort ^[14][33] or cross-sectional studies ^[15][34]. RWesearchers defined cross-sectional as those studies using a single time-point of data for the dietary intake measure (e.g., follow-up), whereas cohort studies were those that used multiple time-points of data and calculated change over time (e.g., change from baseline to follow-up). The Newcastle—Ottawa Scale utilizes a star system to assess the methodological quality of cohort and cross-sectional studies. The cohort tool assigns a maximum of nine stars across three domains: (1) selection of study groups (up to four stars); (2) the comparability of these groups (up to two stars); and (3) assessment of outcomes (up to three stars). The cross-sectional tool assigns a maximum of ten stars across the same three domains: (1) selection of study groups (up to five stars), (2) the comparability of these groups (up to two stars), and (3) assessment of outcomes (up to three stars) ( Supplementary File S3 ).

Pooled quantitative synthesis was not possible due to high heterogeneity across the studies included in the review. An overview of all associations including direction, strength, and favorability, along with the characteristics of the included studies, are summarized, in full, in Table 1 .

For the vote count, studies were counted once for each engagement construct where they provided one or more measures of association. For example, if a study reported two associations of engagement (e.g., time on-site and logins) both would have been included in the vote count synthesis. If there were multiple tests of association reported using the same engagement measure and same dietary outcome in the one study (e.g., multiple associations reported for ‘time on site’ and fruit and vegetable intake), we used the following inclusion criteria to select the association of interest from each study for inclusion in the vote count: If a study had multiple associations using the same engagement measure and same dietary outcome, preference was given to the dietary outcome assessed using the instrument judged by the authors (in the absence of published reliability or validity data) to be most comprehensive. For example, if a study reported two associations including i) ‘time on website’ using an ‘all day’ fruit-and-vegetable screener and ii) ‘time on website’ using a ‘single-item’ fruit-and-vegetable screener, preference was given to the ‘all day screener’ as it is the more comprehensive outcome measure for fruit-and-vegetable intake. If multiple models were presented assessing the association between the same dietary outcome and same engagement measure (e.g., unadjusted and adjusted) we gave preference to the adjusted model. If multiple engagement measures were used and they all assessed the same type of engagement outcome (e.g., time on site) we selected the most complete and inclusive. For example, ‘total time on website’ was given preference to ‘time on a specific website feature’.

3. Key StreDiscussiongth

The aim of the review was to describe the association between engagement with DHIs and improvements in dietary intake. The review included seven studies reported in nine articles. There was considerable heterogeneity in the reporting of both engagement and diet outcomes and the quality of evidence, for a majority of the studies, was considered poor. Overall, the review found mixed evidence supporting an association between DHI usage and dietary intake (five of twelve usage associations were positive, seven were inconclusive). The review found no evidence regarding an association with subjective experience (zero of two associations were inconclusive). Overall, the few studies included in the review, heterogeneity in outcomes and poor quality of evidence limit our ability to draw meaningful conclusions and clearly indicate a need for future research in this area given the wide-held belief that engagement is associated with effectiveness of DHIs.

Like the review by Mclaughlin, we found that logins were the most common engagement outcome reported by included studies. However, unlike the Mclaughlin review, logins were not consistently positively associated with measures of dietary intake and were only significantly associated with intake of fruit, but not ‘vegetables’, ‘calories from sugar-sweetened drinks’, and ‘intake from other food groups’. The findings may provide some early evidence to suggest that the relationship between DHI usage (including that assessed via logins) and behaviour change may differ depending on the target behaviour, at least for dietary outcomes. Additionally, these findings may reflect the effectiveness of DHIs and dietary interventions more broadly, where there is a smaller impact on vegetables compared to fruit [44]. More research is required to examine any differences in the relationship between DHI engagement and health behaviours (physical activity vs diet) and across different dietary habits. Alternatively, the differences in the reported associations of logins between this review and that of Mclaughlin et al. may be due to two of the five included studies reporting multiple tests of association of logins and a range of dietary intake outcomes. Multiple testing of dietary outcomes which may have been targeted by the DHI may produce spurious findings [38,39]. Future research that limits multiple-hypothesis testing (due to multiple dietary outcomes) or more intentionally pre-specifies the primary dietary outcomes to align with the target of the intervention and considers consolidated dietary-intake measures (such as diet quality indexes) in their assessment of association may be warranted.

Consistent with other reviews conceptualising DHI engagement [15] we identified only two studies examining measures of subjective experience. Measures of subjective experience can provide important information on why users engage with DHIs [29] and are hypothesized to predict greater DHI effectiveness [15]. Since publication of the included studies, validated instruments to assess subjective experience have been developed [26]. As such, future research should be undertaken using consistent measures of subjective experience to enable valid and reliable comparisons between studies. This review also identified no studies that used physiological measures of engagement, such as eye tracking, which can often capture which characteristics of the DHI attract attention [29]. For example, eye tracking may capture ‘passive engagement’ or ‘lurking’ whereby users may view, read, and benefit from content posted in forums or on social media posts but do not actively interact with the DHI [29]. These measures also provide objective data on user’s attention, interest, and affect within the DHI and could provide insight into the differential level of engagement for different dietary targets. The lack of studies using physiological measures of engagement is largely consistent with findings of other research in the DHI engagement field [15,29]. As such, future research using broader engagement measures are required to better understand the relationship between DHI engagement and improvements in dietary intake.

A key strength of this review was the comprehensive search strategy, which included screening of 10,653 citations, utilizing published search filters and manual searching in relevant journals and of grey literature. Another strength was that it included measures of subjective experience, a key engagement outcome often overlooked in previous research and suggested to be an important predictor of DHI effectiveness ^[10][15]. Despite this, the review should be interpreted in the context of its limitations. First, the heterogeneity of engagement outcomes in studies precluded meta-analyses, an issue reported in previous reviews of engagement ^[3][21][22][8,24,45]. As such rwesearchers provide a narrative synthesis and rely on methods such as vote counting in order to synthesize study findings. Second, the inclusion of additional databases in the search may have resulted in additional included studies. Finally, all but two studies were rated as ‘poor quality’, studies were primarily downgraded due to their analysis not being described clearly, which was often a result of the association being included as exploratory or process data rather than a primary or secondary study outcome.

4. Conclusions

CThe current reviews addresses an important knowledge gap in the engagement literature and is the first to synthesize the association between DHI engagement and dietary intake. The findings suggest there is some evidence supporting an association with usage, however this was inconsistent. No evidence was found regarding an association with subjective experience. Whilst it has been hypothesized that the modest effects of dietary DHIs are due to poor engagement ^[13][18], the findings do not yet support this and provide little guidance as to which components of engagement to target to enhance the effectiveness of DHIs. Given the reliance on many public health nutrition strategies on DHIs ^[23][24][30,46], a better understanding of the nature of the relationship is a priority for the field. Specifically, the development and application of consistent and comprehensive measures of the multiple dimensions of engagement is required, and the use of more nuanced, mixed-method, and qualitative approaches may be required to better understand the relationship between DHIs and engagement. In particular, it has been hypothesized that the relationship between engagement and behavior change is unlikely to be linear ^[25][37] and that greater engagement may not necessarily yield greater effects on behavior change. Furthermore, we must better understand how relationships are altered by other important contextual factors, including the nature of the DHI, the complexity of the target behavior, and other influences. Ultimately knowing when, and what type of engagement is most important for which behavioral targets, and in what context, will optimize the effects of DHIs for community nutrition improvement.