Skip to main content
BMC is moving to Springer Nature Link. Visit this journal in its new home.

MoveMentor—examining the effectiveness of a machine learning and app-based digital assistant to increase physical activity in adults: protocol for a randomised controlled trial

Abstract

Background

Physical inactivity is prevalent, leading to a high burden of disease and large healthcare costs. Thus, there is a need for affordable, effective and scalable interventions. However, interventions that are affordable and scalable are beset with modest effects and engagement. Interventions that integrate machine learning with real-time data to offer unprecedented levels of personalisation and customisation might offer solutions. The aim of this study is to conduct a randomised controlled trial to evaluate the effectiveness of a machine learning and app-based digital assistant to increase physical activity.

Methods

One hundred and ninety-eight participants will be recruited through Facebook advertisements and randomly allocated to an intervention or control group. Intervention participants will gain access to an app-based physical activity digital assistant that can learn and adapt in real-time to achieve high levels of personalisation and user engagement by virtue of applying a range of machine learning techniques (i.e. reinforcement learning, natural language processing and large language models). The digital assistant will interact with participants in 3 main ways: (1) educational conversations about physical activity; (2) just-in-time personalised in-app notifications (‘nudges’), cues to action encouraging physical activity and (3) chat-based questions and answers about physical activity. Additionally, the app includes adaptive goal setting and an action planning tool. The control group will gain access to the intervention after the last assessment. Outcomes will be measured at baseline, 3 and 6 months. The primary outcome is device-measured (Axivity AX3) moderate-to-vigorous physical activity. Secondary outcomes include app engagement and retention, quality of life, depression, anxiety, stress, sitting time, sleep, workplace productivity, absenteeism, presenteeism and habit strength.

Discussion

The trial presents a unique opportunity to study the effectiveness of a new generation of digital interventions that use advanced machine learning methods to improve physical activity behaviour. By addressing the limitations of existing conversational agents, we aim to pave the way for more effective and adaptable interventions.

Trial registration

Australian New Zealand Clinical Trial Registry ACTRN12624000255583p. Registered on 14 March 2024. https://www.anzctr.org.au/Trial/Registration/TrialReview.aspx?id=387332.

Peer Review reports

Introduction

There is a wealth of evidence regarding the benefits of regular physical activity [1]. Physical activity guidelines recommend a minimum of 150 min of moderate-to-vigorous physical activity per week to prevent the development of cardiovascular disease, diabetes, obesity, cancer and depression [2, 3]. Yet, more than a third of the world’s population does not meet these guidelines [4], causing 9.4% of the total burden of disease and estimated to cost INT$520 billion between 2020 and 2030 through healthcare costs and lost productivity [5, 6]. Therefore, there is an important need to develop and evaluate interventions that can increase physical activity at a population level and at an affordable cost.

Unfortunately, increasing physical activity at a population level has been difficult to achieve. While intensive face-to-face counselling by health professionals is effective, it cannot be implemented at scale due to limited feasibility and high costs [7]. Alternative methods to increasing activity levels at scale (e.g. phone, print, websites, apps) have shown promise, but effect sizes tend to be modest [8,9,10]. As such, the challenge to develop effective, affordable and scalable physical activity interventions remains. In this context, the application of machine learning and artificial intelligence to increase activity levels is increasingly gaining interests by behavioural scientists.

While the concepts of machine learning and artificial intelligence were first established in the 1950 s [11], it has taken more than 60 years for their broad application and implementation in today’s society, due to a lack of computer processing power and speed [12]. Though, right now, we are experiencing nothing short of a revolution in terms of the rapid proliferation and adoption of very powerful machine learning applications (e.g. it took less than 2 months for ChatGTP’s large language model to reach a 100 million users).

Previously, computer-based expert systems have applied basic If–Then decision-rules to provide personal physical activity advice (so-called computer-tailored interventions) [13]. Though these interventions demonstrated superior effectiveness compared to generic or targeted interventions [14], they are not optimally suited to handle the complex and dynamic nature of behaviour change and are not able to provide the right support at the right time (i.e. just-in-time adaptive interventions) [15]. Yet, physical activity interventions that apply machine learning algorithms might be able to overcome the limitations of the previous generation of computer-based interventions.

Some have already examined the use of machine learning to increase physical activity levels [16,17,18], and while the outcomes of these studies have been promising, it is important to point out that the methodological rigour in those studies was low and that the true application of machine learning has been limited. Most of these studies have examined chatbots that have applied off the shelf natural language processing (NLP) but have not implemented machine learning techniques that involved high levels of personalisation or the use of real-time data from activity sensors to deliver just-in-time messages [16,17,18].

Therefore, based on our initial work in this area [19, 20], we published an innovative concept of a sophisticated machine learning-based digital assistant, able to provide highly personalised physical activity advice, delivered just-in-time based on real-time data [21]. After completing focus group research [22], co-creation workshops, software development and extensive beta-resting, we have now developed a digital assistant that is able to dynamically and proactively nudge users into being more physically active. Our intervention applies machine learning techniques rarely applied to increase physical activity to date (i.e. reinforcement learning). In addition, the intervention also uses NLP, along with an integrated large language model (i.e. Google’s Gemini) that can formulate coherent and relevant answers to any physical activity-related questions participants may have. Now that the development phase has been completed, we are ready to evaluate the effectiveness of this new intervention. For this evaluation, we will focus on workplace-based employees, as workplaces offer a captive audience to implement interventions to large populations for potential future translation efforts [23]. Moreover, workplace interventions can achieve outcomes that go beyond health benefits, such as improved productivity, workplace morale, reputation and lower absenteeism [24]. Therefore, the aim of this study is to conduct a randomised controlled trial to examine the effectiveness of a machine learning and app-based digital assistant to increase physical activity in office-based employees. We hypothesise that participants receiving the intervention will significantly increase their moderate to vigorous physical activity at 3- and 6-month follow-ups compared to participants allocated to a no intervention control group.

Methods

Trial design

The MoveMentor trial is a parallel group, two-arm, superiority trial with a 1:1 allocation ratio to investigate the effectiveness of an app-based digital assistant to increase physical activity compared to a no intervention control group. Given the novelty of the intervention, we first need to establish the effectiveness of the intervention against a true control group, prior to comparing it to alternative interventions. The control group will be offered access to the intervention after the final assessment has been completed. The study setting is ‘home-based’, as there will be no face-to-face contact with participants, and all communication will occur via e-mail or phone. Primary and secondary outcomes will be assessed at baseline, 3 months and 6 months. The study has received ethics approval from the Human Research Ethics Committee (HREC) of the Central Queensland University (0000024786) and complies with the Helsinki Declaration. While not anticipated, any adverse events will be reported immediately to the HREC and managed according to their instructions and reported in publications resulting from this study. Likewise, ethical approval will be sought prior to any protocol amendments should they be needed. The Central Queensland University is the sponsor for this trial. There are no plans to use participants’ data in any other studies. This study protocol has been prepared in accordance with SPIRIT guidelines (see Additional file 1 and Table 1) and study outcomes will be reported according to CONSORT guidelines [25, 26].

Table 1 SPIRIT figure

Participants

One hundred and ninety-eight participants will be recruited through Facebook advertisements. As response rates to Facebook advertisements are lower in men [27], more funds will be spent on advertisements targeting men, as to obtain a reasonably balanced sample. Recruitment rates will be closely tracked throughout the recruitment phase to determine how much should be spent on advertisements for each gender. All advertisements will direct interested individuals to a Qualtrics survey for screening. After potential participants have completed the screening questionnaire and are deemed eligible, they will be provided with a detailed study information sheet. If those deemed eligible are still interested in participating, they will be asked to provide contact details and informed consent. Those deemed ineligible will be informed of why, but they will not be able to continue.

Eligible participants will be:

  • Over 18 years of age

  • Live anywhere in Australia

  • Speak and read the English language

  • Engaged in full-time office-based employment

  • Have a smartphone (i.e. Apple or Android) with internet access

  • Be physically inactive. Participants will be asked: ‘How many days per week do you engage in at least 30 min of activity of a moderate intensity or physical activity higher? For example, brisk walking, swimming, tennis, etc.?’ Anyone reporting more than 2 days per week will be excluded.

  • Have no impairments limiting an increase in physical activity

  • Did not participate in any physical activity programs in the past 12 months

  • Not own or have used a physical activity tracker for at least 12 months

  • Not be pregnant

  • Have a body mass index over 17.5

Procedure

When contact details of eligible people are received through the Qualtrics survey, a project officer will make contact by phone to verify the information provided, make sure they are aware of and understand all study requirements and that they still agree to participate. Subsequently, they will be e-mailed a link to access to an online baseline survey (administered via Qualtrics). Participants will also be sent an accelerometer-based activity monitor using registered mail, as well as instructions on how to use the device, a 7-day monitoring log and a pre-paid return post bag. They will be asked to wear the activity monitor on the non-dominant wrist for 7 consecutive days and nights from the day of receipt. When the accelerometer is returned to the laboratory, it will be checked for valid wear time. Potential participants providing less than four valid monitoring days (≥ 10 h of wear time) will be excluded from the study. The same measurement procedures (online survey and 7-day accelerometer) will be repeated at 3 and 6 months (though participants with incomplete data will not be excluded at this stage). Participants will be rewarded with a $50 AUD shopping voucher for each completed assessment. Participants can withdraw from the study without any consequence at any stage. Whenever needed, reminder e-mails and phone calls will be used to encourage participants to complete assessments.

In accordance with the approved study data management plan (DMP_2790), data from all sources will be securely stored for a minimum of 10 years on a university server dedicated to research data and regularly backed up. Only study investigators named on this publication will have access to the final trial dataset. Given the small scale of this study, no data monitoring committee will be established beyond the involvement of the study investigators named on this publication. Participants’ identity will be protected at all times; only de-identified and aggregated data will be disseminated in publications and reports. A plain English statement summarising trial outcomes will be shared with participants that indicated an interest. Publications resulting from this study will abide by ICMJE authorship guidelines and not involve professional writers.

When the initial activity monitoring period and baseline survey have been completed, participants will be randomised into one of two groups using a randomly generated sequence created by CV within Microsoft Excel (version 16.89.1, Microsoft Corporation, Redmond, WA). The column containing the randomised sequence will be concealed in the spreadsheet and will only be unmasked row-by-row by the study manager (DH) after a new participant has been added to the spreadsheet. After randomisation, intervention group participants will be sent a Fitbit Inspire 2 activity tracker (Fitbit Inc, San Francisco, CA) as part of the intervention. They will be asked to download the Fitbit app onto their smartphone, connect their Fitbit tracker with this app and wear the Fitbit for at least 7 days. The Fitbit is worn on a participant’s preferred wrist, with the dominant or non-dominant wrist preference recorded in the Fitbit app on setup. After this, they will be asked to download the digital assistant smartphone app (i.e. MoveMentor) onto their smartphone and grant permission for MoveMentor to access their Fitbit data. Once everything is set up, intervention group participants will be asked to wear their Fitbit as much as possible and actively engage with the MoveMentor app during the 6-month intervention period. Intervention participants’ use of MoveMentor will be monitored for the entire 6-month study duration. Intervention participants who do not download or use MoveMentor will be encouraged to download and/or use to use the app via e-mail. Given the behavioural nature of the study, aspects of concomitant care are not applicable in this study. At the end of the study, intervention group participants will be able to keep the Fitbit activity tracker and access to the MoveMentor app will continue to be available via the Apple App and Google Play stores for all study participants. The flowchart in Fig. 1 shows an overview of the study procedures.

Fig. 1
figure 1

Flowchart

Intervention

A detailed description of the intervention concept has been published elsewhere [21]; this includes in-depth information on how machine learning aspects have been applied. Briefly, the intervention consists of a stand-alone smartphone app (for both Apple and Android operating systems) named MoveMentor (see Fig. 2). The app works in conjunction with physical activity trackers (e.g. Fitbit). The main feature of the app is the physical activity digital assistant (i.e. chatbot or conversational agent). While the name of the digital assistant in the app stores is MoveMentor, users can change the name inside the app and select one of three main characters (i.e. MoveMentor (default), MoveMate or MyPT) or choose their own name and image for the digital assistant.

Fig. 2
figure 2

Image of the MoveMentor app homepage

The digital assistant interacts with participants in 3 main ways: (1) educational conversations; (2) nudges (via smartphone push notifications) and (3) chat-based questions and answers (Q&A). The conversations provide participants with educational content in relation to physical activity delivered through interactive ‘conversations’ or ‘chats’. The aim of the conversations is to encourage participants to become more physically active and meet the national physical activity guidelines (i.e. 150 to 300 min of moderate-to-vigorous intensity physical activity), as well as to support them in overcoming any difficulties in becoming more active [28]. The conversations are supported by Google’s Dialogue Flow and use natural language processing to enable a more natural flow for the conversation. Example conversation topics include ‘Developing a lasting activity habit’, ‘What stops you from being active?’, ‘Make the most of your surroundings’, ‘Being active with others’ and ‘Building confidence for being active’. During the conversations, the MoveMentor will ask participants physical activity questions and participants will be provided with personalised advice, depending on the answers they provide. Additional resources are provided at the end of each conversation for further reading, watching or listening. The conversations are designed to last no longer than 5 or 10 min. A list of conversations is available for participants on the homepage of the app; however, participants will also receive ‘nudges’ (i.e. notifications) to engage in these conversations. Participants will be able to go through a conversation more than once if they want to do so. The conversations have been informed by the Self-Determination and Social Cognitive Theories [29, 30].

Based on the ‘Nudge’ and ‘COM-B’ theories (Capability-Opportunity-Motivation and Behaviour), participants will also be nudged into action [31, 32]. The nudges (delivered as a smartphone push notification) should be thought of as a Just-In-Time Adaptive Intervention (JITAI), which uses real-time data to dynamically address user needs by providing the right cue to action at the right time to assist with long-term habit formation [15]. The timing and content of the nudges are determined by machine learning algorithms that learn what works for whom under what circumstances through trial and error. The machine learning algorithm is trained to make the nudges more personally relevant over time and will use real-time feedback from activity trackers to learn what nudges work and do not work (i.e. an increase in physical activity following nudges or not). The nudges that work will be reinforced, and those that do not will be phased out. More specifically, a ‘contextual bandit’ approach is applied to deliver the right nudges at the right time; this is a type of reinforcement learning [33, 34]. Contextual data will be derived from participants’ app preferences and settings (derived using an onboarding survey), participant interactions with the app and digital assistant, physical activity data from the activity trackers, nudge ratings (i.e. like or dislike) and weather data (to determine whether indoor or outdoor activity is recommended based on the participants’ location at that specific time). An extensive library of nudges has been developed to send to users through push notifications. The library consists of a range of nudge categories from which nudges are selected in a random pattern. However, a nudge that has been used before will not be used again unless all other nudges within a category have been used as well. Examples of nudge categories are ‘daily morning check-in’, ‘weekly action plan review’, ‘activity suggestion’ (e.g. inspirational, based on known participant characteristics, based on weather), ‘motivational’ (i.e. to encourage meeting their daily activity goal), ‘educational’ (i.e. physical activity facts, tips and tricks), ‘streaks’ (i.e. achieving daily goals repeatedly), ‘milestones’ (e.g. recording over a million steps) and more. The reinforcement learning algorithms have been trained with the help of 300 beta-testers over a 6-month period to improve their performance.

Finally, participants can engage with the digital assistant by asking it physical activity-related questions. Answers to questions that have not been pre-programmed (e.g. I have a muscle pain in my calf, how can I still be active?) are being delivered by Gemini (Alphabet Inc, Mountain View, CA), Google’s machine learning and natural language generation model (it is a large language model comparable to ChatGPT). To generate the best answer, additional contextual information will be sent to Gemini. This will include previous conversations, the user’s nominated username, recent physical activity, local weather and other information provided during the usage of the app. The answer will be generated by the Gemini algorithms.

Beyond interactions with the digital assistant and personalisation features, the MoveMentor app includes additional features, of which adaptive goal setting and action planning are the most important. Every morning, participants receive a daily activity goal (expressed as steps or activity minutes, as per participant preference). This goal is based on whether they met their activity goal during the previous day, as well as the average physical activity conducted in the last 4 weeks. The goal is designed to slowly increase over time, aligned with how well participants are performing, until it reaches a participant’s ‘long-term activity’ goal (defined as part of the action planning process), at which point the goal stops increasing. If participants are consistently not meeting their daily activity goal, then the goal will gradually decrease, to better align with the participant’s activity journey.

Participants are encouraged to engage in an action planning process [35]. This conversation is designed to help participants formulate a weekly action plan. If participants are to achieve their long-term goal, they need concrete short-term action. As such, the action plan helps participants define what physical activities they will engage in, where they will do them, how many times per week, what days of the week, what time of day, how long activity sessions will last and identify any people they may be active together with. To be effective, action plans need to be updated regularly and the digital assistant will send out a weekly reminder to review participants’ action plans.

Control group participants will not receive access to the intervention until after the 6-month waiting period. They will not receive a Fitbit activity monitor.

Measures

Primary outcome

Device-measured moderate-to-vigorous physical activity: Participants will be asked to wear an Axivity AX3 tri-axial accelerometer (https://axivity.com/product/ax3) on their non-dominant wrist for 7 consecutive days during waking and sleeping hours, except when swimming, bathing or participating in contact sports. Participants will also be asked to complete an activity monitor log detailing times the monitor was removed, time they went to bed and wake up time. The Axivity AX3 has demonstrated good concurrent validity [36].

Raw accelerometer data collected at 50 Hz will be processed into physical activity metrics using a machine-learned random forest classifier specifically designed and validated for assessing movement behaviours in free living adults [37]. The random forest classification model uses features in the raw acceleration signal to classify each 10-s window (epoch) as sedentary (lying or sitting still), stationary plus (active sitting, standing still, active standing), walking or running. Daily time spent in moderate-to-vigorous intensity physical activity (MVPA) will be calculated using the approach established by Ahmadi et al. in their analysis of accelerometer data from the UK biobank study [38]. Windows classified as walking with normalised gravitation units (milli g) ≥ 100 milli g and < 400 milli g will be considered moderate intensity physical activity (3–6 METs). Windows classified as walking with normalised gravitational units ≥ 400 milli g will be considered vigorous intensity physical activity (≥ 6 METs), while all windows classified as running will be considered vigorous intensity physical activity (≥ 6 METs). Non-wear periods will be identified and differentiated from sleep using the methods described by Ahmadi and colleagues [39]. The participants’ accelerometer data will be included in the analyses if they have ≥ 4 days (with at least 1 weekend day) in which daily wear time is ≥ 10 h [40]. Device-measured physical activity will be reported as the average minutes of MVPA per day or week for each time point and group.

Secondary outcomes

Health behaviours

  • Device-measured sedentary time and sleep will also be assessed using the Axivity AX3 tri-axial accelerometer. Sleep periods will be detected using the Van Hees et al. heuristic [41].

  • Self-reported physical activity will also be assessed. The Active Australia Survey provides contextual physical activity information as it assesses the frequency and duration (in minutes per week) of walking for transport, walking for recreation, moderate physical activity and vigorous physical activity [42]. The Active Australia Survey has acceptable test–retest reliability and validity in the Australian adult population and has been documented as a useful evaluative tool for detecting intervention related change in physical activity [43, 44].

  • Self-reported sitting time will be measured using the 10-item ‘Workforce Sitting Questionnaire’ [45]. All participants will report time (hours/minutes) spent sitting on usual working and non-working days in relation to work, transport, TV use, computer use and other leisure-time sitting. One question also measures the number of days participants usually work in a week. This survey has demonstrated adequate test–retest reliability and validity [45].

  • Self-reported sleep behaviour will be assessed using items from both the Behavioural Risk Factor Surveillance Screening Sleep Module [46] and the valid and reliable Pittsburgh Sleep Quality Index [47]. This study assesses average hours of sleep obtained, snoring behaviour, unintentionally falling asleep during the day, nodding off while driving, whether one feels rested and levels of sleep quality. Additionally, an objective measure of sleep will be collected using the Axivity accelerometer. Total sleep time, sleep efficiency, number of awakenings, sleep onset and sleep offset will be extracted from the accelerometer data using established protocols [39].

Work-related outcomes

  • The Health and Work Questionnaire (HWQ), a survey for assessing workplace productivity, consists of 24 items, with each item rated on a 10-point Likert scale (e.g. ‘very dissatisfied’ to ‘very satisfied’) [48]. The questions assess work quality, quantity and efficiency, as well as impatience, concentration/focus, work satisfaction and non-work satisfaction. The HWQ shows very good internal consistency reliability [48].

  • The World Health Organization’s Heath and Work Performance Questionnaire (HPQ), will be used to assess absenteeism and presenteeism [49]. Eight items are used to assess absenteeism and 3 items are used to assess presenteeism. Good concordance was found between the HPQ and other productivity indicators [49].

  • The Maslach Burnout Inventory will assess various aspects of burnout syndrome [50]. It has a total of 22 items that are scored on 7-point scales and contains 3 subscales: emotional exhaustion, depersonalisation and personal accomplishment. Various psychometric analyses showed that the scale has both high reliability and validity as a measure of burnout [50].

  • The Employee Wellbeing Scale: This survey, with demonstrated validity and reliability [51], assesses employee wellbeing using 18 items on 7-point Likert scales (scored: 1 = strongly disagree; 7 = strongly agree). The survey assesses wellbeing across three dimensions: life wellbeing, work wellbeing and psychological wellbeing.

Mental health and quality of life

  • Quality of life will be measured through the Short-Form Health Survey (SF-12) which includes questions on physical and mental health. The SF-12 is widely used, valid and reliable [52].

  • Depression, anxiety and stress: The 21-item Depression, Anxiety and Stress Scale (DASS21) will be used to assess mental health outcomes [53]. The DASS21 provides a quantitative measure of distress symptom severity; it is not a measure of clinical diagnosis. Three 7-item subscales measure depression, anxiety and stress. These scales have demonstrated acceptable validity and reliability in community samples of adults [53].

  • The Satisfaction with Life Scale is a 5-item measure of an individual’s global evaluation of their life [54]. Participants complete the measure on a 1 to 7 Likert-type response scale. Scores can range between 5 and 35, with higher scores indicating greater life satisfaction. The scale has been widely used, with high reliability and validity [55].

  • The Positive and Negative Affect Schedule—short form (I-PANAS-SF) is a 10-item Positive and Negative Affect Scale [56]. The scale measures an individual’s experiences of positive and negative affect with two 5-item subscales. Participants complete the measure on a 1 to 5 Likert-type response scale. Scores can range between 5 and 25 for each subscale, with higher scores indicating greater experience of positive or negative emotion. The scale is reliable, valid and has demonstrated cross-cultural validity [57].

  • Habit strength will be assessed using the 4-item Self-Reported Behavioural Automaticity Index [58], which is a subscale of the Self-Reported Habit Index [59]. Participants are asked to what extent they agreed with the items, ‘Physical activity is something I do… (1) automatically, (2) without having to consciously remember, (3) without thinking, (4) before I realise I’m doing it’, using response scales from 1 (strongly disagree) to 7 (strongly agree). The habit strength score is calculated as the mean of the 4 items. This scale was found to be reliable and sensitive to the effects of habit on behaviour [58].

Intervention use and usability

  • App usability and usefulness: The overall usability of the app will be investigated using the System Usability Scale [60]. This 10-item scale has demonstrated acceptable reliability and validity for measuring the usability of websites and apps [60]. Additionally, there will be 5 items on how useful participants found the app in supporting them in becoming more active. These outcomes will only be measured at the 3-month time point.

  • App usage statistics: App usage will be measured using the Google Analytics Firebase traffic analysis platform (Alphabet Inc, Mountain View, CA), as well as by monitoring user-generated content. Features monitored will include (but are not limited to) the number of times the app was opened, time in the app, app pages visited, number of action plans completed, number of completed conversations, questions asked in chat, amount of nudge feedback provided, physical activity recorded through the app (i.e. activity minutes and steps) and activity milestones achieved (e.g. times the daily goal was met). App usage will be monitored for the entire study duration.

Demographics

The following demographic factors will be assessed: age, sex, weight, height, marital status (single, widowed, divorced, separated, married, in a relationship), ethnicity (Caucasian, Aboriginal or Torres Strait Islander, African, Asian, other), number of years of education, occupational category (e.g. manager, professional, sales), income, postcode, residential environment (e.g. rural area), Accessibility/Remoteness Index of Australia (ARIA) [61] and presence of chronic diseases (e.g. hypertension, stroke, arthritis, depression). Additionally, alcohol consumption will be assessed using the Alcohol Use Disorders Identification Test (AUDIT-C). It is a 3-item test for heavy drinking and/or active alcohol abuse or dependence, with demonstrated validity and reliability [62]. Smoking will be assessed using a single item to classify participants as either a current smoker or non-smoker. Diet will be assessed using four items that evaluated the daily frequency of fruit and vegetable consumption and the number of times soft drinks and fast foods were consumed in the previous 7 days [63]. An overview of all measures and when they are being assessed is provided in Table 2.

Table 2 Overview of measures, instruments and when they are being assessed

Sample size

Sample size for the study is based on the primary outcome of change in device-measured MVPA (min/day). Several reviews of app-based physical activity interventions suggest that most studies have small to medium effects on change in physical activity and have high dropout rates, up to approximately 30% [64, 65]. Hence, to detect a small to medium difference in the change in daily MVPA between groups (d = 0.45; corresponding with a minimal detectable difference of 10 min/day), at the study’s primary time point of 6 months, 76 participants per group will be required to achieve 80% power using an alpha level of 0.05. The number of participants per group will be inflated by 30% (99 participants per group) to account for drop out. Hence, a total of 198 participants will be recruited into the RCT.

Statistical analyses

Data will be analysed by a statistician blinded to group allocation, using generalised linear mixed models (GLMM), including fixed effects for group, time, the group-by-time interaction and a random subject effect [66]. GLMM analyses examine changes from baseline to each follow-up time point for each intervention group compared to the control group while accounting for within-person nesting of variables across time [66]. Random effects mixed modelling will also be used to explore potential changes in secondary outcomes. Planned subgroup analyses include examining any moderating effects of sociodemographic variables (e.g. sex, age, income) and app usage as potential mediator for increases in physical activity levels. Data will be tested for linearity and normality to ensure the proper statistical techniques are applied. The most appropriate approach to handling missing data will be applied depending on the pattern and amount of the missing data. However, if possible, intention-to-treat including all participants with valid baseline data and multiple imputation by chained equations will be used to handle missing data [67]. The analyses will be conducted using SPSS v30.

Discussion

The current trial presents a unique opportunity to study the effectiveness of a new generation of digital interventions that use reinforcement learning (i.e. contextual bandit) in combination with real-time data to deliver a highly personalised app-based digital assistant for promoting physical activity [23]. By addressing the limitations of existing conversational agents and applying advanced machine learning techniques, we aim to pave the way for more effective and adaptable interventions. There is an ongoing proliferation of health and physical activity smartphone apps, yet very little work has been carried out to study the utility and effectiveness of these population-targeted apps [68]. Soon, many more of these apps will also be applying various form of machine learning techniques to influence health behaviour. As such, our work will contribute to understanding how effective these approaches are, and what potential downsides might be. We expect that the current study’s findings will provide increased insights into the benefits of applying machine learning in engaging and retaining participants in app-based interventions, something previous interventions have struggled with [69, 70]. Higher participant engagement and retention have been shown to result in higher levels of behaviour change, and ultimately this will lead to improved health outcomes [70].

There are several strengths to this trial; for example, our trial will have more participants and measure intervention outcomes over a longer period than most comparable interventions in this space [18,19,20,21]. Additionally, our trial applies a robust study design, rigorous participant screening, device-measured physical activity outcomes, numerous measures to assess secondary outcomes, standardised exposure and contact across groups, and sufficient analytical power to observe between group changes. The lack of face-to-face contact with participants aligns with the nature of real-world digital interventions and allows the recruitment of participants anywhere in Australia, which enhances the generalisability of the study findings. However, a potential downside of this is a lower feeling of accountability towards study participation and higher levels of attrition. Nevertheless, this study and the intervention it aims to evaluate are well-positioned to generate new knowledge and advance the field in this space.

Trial status

Protocol version 1.0, September 2024. Currently, the trial is ongoing and recruitment of participants continues. Recruitment began on 5 August 2024 and was expected to be completed by late October 2024.

Protocol version 2.0, February 2025. Currently, the trial is ongoing and recruitment was completed on 25 November 2024.

Protocol version 3.0, June 2025. Currently, the trial is ongoing for a small number of participants, i.e. those that were recruited early on have already completed their participation.

Data availability

Upon study completion, only named study investigators will have access to the final trial dataset. While the data will not be publicly available, reasonable requests for access will be considered by the lead investigator (CV) in line with (additional) ethical approvals.

References

  1. Warburton DE, Nicol CW, Bredin SS. Health benefits of physical activity: the evidence. CMAJ. 2006;174(6):801–9.

    Article  PubMed  PubMed Central  Google Scholar 

  2. Blair SN, Morris JN. Healthy hearts—and the universal benefits of being physically active: physical activity and health. Ann Epidemiol. 2009;19(4):253–6.

    Article  PubMed  Google Scholar 

  3. Bull FC, Al-Ansari SS, Biddle S, Borodulin K, Buman MP, Cardon G, Carty C, Chaput JP, Chastin S, Chou R, Dempsey PC. World Health Organization 2020 guidelines on physical activity and sedentary behaviour. Br J Sports Med. 2020;54(24):1451–62.

    Article  PubMed  Google Scholar 

  4. World Health Organization. Global status report on physical activity 2022: country profiles. Geneva: World Health Organization; 2022.

    Google Scholar 

  5. Lee IM, Shiroma EJ, Lobelo F, Puska P, Blair SN, Katzmarzyk PT. Effect of physical inactivity on major non-communicable diseases worldwide: an analysis of burden of disease and life expectancy. The lancet. 2012;380(9838):219–29.

    Article  Google Scholar 

  6. Santos AC, Willumsen J, Meheus F, Ilbawi A, Bull FC. The cost of inaction on physical inactivity to public health-care systems: a population-attributable fraction analysis. Lancet Glob Health. 2023;11(1):e32–9.

    Article  CAS  PubMed  Google Scholar 

  7. Richards J, Hillsdon M, Thorogood M, Foster C. Face‐to‐face interventions for promoting physical activity. Cochrane Database Syst Rev. 2013(9). https://doi.org/10.1002/14651858.CD010392.pub2.

  8. Vandelanotte C, Müller AM, Short CE, Hingle M, Nathan N, Williams SL, Lopez ML, Parekh S, Maher CA. Past, present, and future of eHealth and mHealth research to improve physical activity and dietary behaviors. J Nutr Educ Behav. 2016;48(3):219–28.

    Article  PubMed  Google Scholar 

  9. Schoeppe S, Alley S, Van Lippevelde W, Bray NA, Williams SL, Duncan MJ, Vandelanotte C. Efficacy of interventions that use apps to improve diet, physical activity and sedentary behaviour: a systematic review. Int J Behav Nutr Phys Act. 2016;13:1–26.

    Article  Google Scholar 

  10. Singh B, Ahmed M, Staiano AE, Gough C, Petersen J, Vandelanotte C, Kracht C, Huong C, Yin Z, Vasiloglou MF, Pan CC. A systematic umbrella review and meta-meta-analysis of eHealth and mHealth interventions for improving lifestyle behaviours. npj Digital Medicine. 2024;7(1):179.

  11. Fradkov AL. Early history of machine learning. IFAC-PapersOnLine. 2020;53(2):1385–90.

    Article  Google Scholar 

  12. Stephens M. A learning revolution. Phys World. 2019;32(3):45.

    Article  Google Scholar 

  13. Broekhuizen K, Kroeze W, van Poppel MN, Oenema A, Brug J. A systematic review of randomized controlled trials on the effectiveness of computer-tailored physical activity and dietary behavior promotion programs: an update. Ann Behav Med. 2012;44(2):259–86.

    Article  PubMed  Google Scholar 

  14. Short CE, James EL, Plotnikoff RC, Girgis A. Efficacy of tailored-print interventions to promote physical activity: a systematic review of randomised trials. Int J Behav Nutr Phys Act. 2011;8:1–3.

    Article  Google Scholar 

  15. Nahum-Shani I, Smith SN, Spring BJ, Collins LM, Witkiewitz K, Tewari A, Murphy SA. Just-in-time adaptive interventions (JITAIs) in mobile health: key components and design principles for ongoing health behavior support. Ann Behav Med. 2018;18:1–7.

    Google Scholar 

  16. Singh B, Olds T, Brinsley J, Dumuid D, Virgara R, Matricciani L, Watson A, Szeto K, Eglitis E, Miatke A, Simpson CE. Systematic review and meta-analysis of the effectiveness of chatbots on lifestyle behaviours. npj Digital Medicine. 2023;6(1):118.

  17. Oh YJ, Zhang J, Fang ML, Fukuoka Y. A systematic review of artificial intelligence chatbots for promoting physical activity, healthy diet, and weight loss. Int J Behav Nutr Phys Act. 2021;18:1–25.

    Article  Google Scholar 

  18. Luo TC, Aguilera A, Lyles CR, Figueroa CA. Promoting physical activity through conversational agents: mixed methods systematic review. J Med Internet Res. 2021;23(9):e25486.

    Article  PubMed  PubMed Central  Google Scholar 

  19. To QG, Green C, Vandelanotte C. Feasibility, usability, and effectiveness of a machine learning–based physical activity chatbot: quasi-experimental study. JMIR Mhealth Uhealth. 2021;9(11):e28577.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Maher CA, Davis CR, Curtis RG, Short CE, Murphy KJ. A physical activity and diet program delivered by artificially intelligent virtual health coach: proof-of-concept study. JMIR Mhealth Uhealth. 2020;8(7):e17558.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Vandelanotte C, Trost S, Hodgetts D, Imam T, Rashid M, To QG, Maher C. Increasing physical activity using an just-in-time adaptive digital assistant supported by machine learning: a novel approach for hyper-personalised mHealth interventions. J Biomed Inform. 2023;1(144): 104435.

    Article  Google Scholar 

  22. Vandelanotte C, Hodgetts D, Peris DLIHK, Karki A, Maher C, Imam T, Rashid M, To QG, Trost S. Perceptions and expectations of an artificially intelligent physical activity digital assistant – A focus group study. Appl Psychol: Health Well-Being. 2024;16:2362–80.

  23. Quintiliani L, Sattelmair J, Sorensen G. The workplace as a setting for interventions to improve diet and promote physical activity. Geneva: World Health Organization; 2007. p. 1–36.

    Google Scholar 

  24. Conn VS, Hafdahl AR, Cooper PS, Brown LM, Lusk SL. Meta-analysis of workplace physical activity interventions. Am J Prev Med. 2009;37(4):330–9.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Chan AW, Tetzlaff JM, Gøtzsche PC, Altman DG, Mann H, Berlin JA, Dickersin K, Hróbjartsson A, Schulz KF, Parulekar WR, Krleža-Jerić K. SPIRIT 2013 explanation and elaboration: guidance for protocols of clinical trials. BMJ. 2013;9:346.

    Google Scholar 

  26. Altman DG, Schulz KF, Moher D, Egger M, Davidoff F, Elbourne D, Gøtzsche PC, Lang T, Consort Group. The revised CONSORT statement for reporting randomized trials: explanation and elaboration. Annals of internal medicine. 2001;134(8):663–94.

  27. Northcott C, Curtis R, Bogomolova S, Olds T, Vandelanotte C, Plotnikoff R, Maher C. Evaluating the effectiveness of a physical activity social media advertising campaign using Facebook, Facebook Messenger, and Instagram. Translational behavioral medicine. 2021;11(3):870–81.

    Article  PubMed  Google Scholar 

  28. Brown W, Bauman A, Bull F, Burton N. Development of evidence-based physical activity recommendations for adults (18–64 years). Report prepared for the Australian Government Department of Health, 2012.

  29. Deci EL, Ryan RM. Self-determination theory. Handbook of theories of social psychology. 2012;1(20):416–36.

    Article  Google Scholar 

  30. Bandura A. Social cognitive theory: an agentic perspective. Annu Rev Psychol. 2001;52(1):1–26.

    Article  CAS  PubMed  Google Scholar 

  31. Michie S, Van Stralen MM, West R. The behaviour change wheel: a new method for characterising and designing behaviour change interventions. Implement Sci. 2011;6:1–2.

    Article  Google Scholar 

  32. Forberger S, Reisch L, Kampfmann T, Zeeb H. Nudging to move: a scoping review of the use of choice architecture interventions to promote physical activity in the general population. Int J Behav Nutr Phys Act. 2019;16:1–4.

    Article  Google Scholar 

  33. Bouneffouf D, Bouzeghoub A, Gançarski ALA, contextual-bandit algorithm for mobile context-aware recommender system. In Neural information processing: 19th international conference, ICONIP,. Doha, Qatar, November 12–15, 2012, Proceedings, Part III 19 2012. Berlin Heidelberg: Springer; 2012. p. 324–31.

    Google Scholar 

  34. Zhu F, Guo J, Li R, Huang J. Robust actor-critic contextual bandit for mobile health (mhealth) interventions. In Proceedings of the 2018 ACM international conference on bioinformatics, computational biology, and health informatics 2018 (pp. 492–501).

  35. Rebar AL, Williams R, Short CE, Plotnikoff R, Duncan MJ, Mummery K, Alley S, Schoeppe S, To Q, Vandelanotte C. The impact of action plans on habit and intention strength for physical activity in a web-based intervention: is it the thought that counts? Psychol Health. 2023;28:1–21.

    Google Scholar 

  36. Hedayatrad L, Stewart T, Duncan S. Concurrent validity of ActiGraph GT3X+ and Axivity AX3 accelerometers for estimating physical activity and sedentary behavior. Journal for the Measurement of Physical Behaviour. 2020;4(1):1–8.

    Article  Google Scholar 

  37. Pavey TG, Gilson ND, Gomersall SR, Clark B, Trost SG. Field evaluation of a random forest activity classifier for wrist-worn accelerometer data. J Sci Med Sport. 2017;20(1):75–80.

    Article  PubMed  Google Scholar 

  38. Ahmadi MN, Hamer M, Gill JM, Murphy M, Sanders JP, Doherty A, Stamatakis E. Brief bouts of device-measured intermittent lifestyle physical activity and its association with major adverse cardiovascular events and mortality in people who do not exercise: a prospective cohort study. The Lancet Public Health. 2023;8(10):e800–10.

    Article  PubMed  Google Scholar 

  39. Ahmadi MN, Nathan N, Sutherland R, Wolfenden L, Trost SG. Non-wear or sleep? Evaluation of five non-wear detection algorithms for raw accelerometer data. J Sports Sci. 2020;38(4):399–404.

    Article  PubMed  Google Scholar 

  40. Trost SG, Mciver KL, Pate RR. Conducting accelerometer-based activity assessments in field-based research. Med Sci Sports Exerc. 2005;37(11):S531.

    Article  PubMed  Google Scholar 

  41. Van Hees VT, Sabia S, Jones SE, Wood AR, Anderson KN, Kivimäki M, Frayling TM, Pack AI, Bucan M, Trenell MI, Mazzotti DR. Estimating sleep parameters using an accelerometer without sleep diary. Sci Rep. 2018;8(1):12975.

    Article  PubMed  PubMed Central  Google Scholar 

  42. Australian Institute of Health and Welfare. The Active Australia Survey: a guide and manual for implementation, analysis and reporting. Canberra: Australian Institute of Health and Welfare; 2003.

    Google Scholar 

  43. Brown WJ, Trost SG, Bauman A, Mummery K, Owen N. Test-retest reliability of four physical activity measures used in population surveys. J Sci Med Sport. 2004;7(2):205–15.

    Article  CAS  PubMed  Google Scholar 

  44. Reeves MM, Marshall AL, Owen N, Winkler EA, Eakin EG. Measuring physical activity change in broad-reach intervention trials. J Phys Act Health. 2010;7(2):194–202.

    Article  PubMed  Google Scholar 

  45. Chau JY, Van Der Ploeg HP, Dunn S, Kurko J, Bauman AE. A tool for measuring workers’ sitting time by domain: the Workforce Sitting Questionnaire. Br J Sports Med. 2011;45(15):1216–22.

    Article  PubMed  Google Scholar 

  46. Centers for Disease Control and Prevention. Behavioral Risk Factor Surveillance System survey questionnaire. Atlanta, Georgia: US Department of Health and Human Services, Centers for Disease Control and Prevention; 2009.

  47. Buysse DJ, Reynolds CF III, Monk TH, Berman SR, Kupfer DJ. The Pittsburgh Sleep Quality Index: a new instrument for psychiatric practice and research. Psychiatry Res. 1989;28(2):193–213.

    Article  CAS  PubMed  Google Scholar 

  48. Shikiar R, Halpern MT, Rentz AM, Khan ZM. Development of the Health and Work Questionnaire (HWQ): an instrument for assessing workplace productivity in relation to worker health. Work. 2004;22(3):219–29.

    PubMed  Google Scholar 

  49. Kessler RC, Barber C, Beck A, Berglund P, Cleary PD, McKenas D, Pronk N, Simon G, Stang P, Ustun TB, Wang P. The World Health Organization health and work performance questionnaire (HPQ). J Occup Environ Med. 2003;45(2):156–74.

    Article  PubMed  Google Scholar 

  50. Maslach C, Jackson SE. The measurement of experienced burnout. J Organ Behav. 1981;2(2):99–113.

    Article  Google Scholar 

  51. Zheng X, Zhu W, Zhao H, Zhang C. Employee well-being in organizations: theoretical model, scale development, and cross-cultural validation. J Organ Behav. 2015;36(5):621–44.

    Article  Google Scholar 

  52. Ware JE, Kosinski M, Keller S. SF-36 physical and mental health summary scales. A user’s manual. 2001;1994.

  53. Crawford JR, Henry JD. The Depression Anxiety Stress Scales (DASS): normative data and latent structure in a large non-clinical sample. Br J Clin Psychol. 2003;42(2):111–31.

    Article  PubMed  Google Scholar 

  54. Diener ED, Emmons RA, Larsen RJ, Griffin S. The satisfaction with life scale. J Pers Assess. 1985;49(1):71–5.

    Article  CAS  PubMed  Google Scholar 

  55. Pavot W, Diener E. The satisfaction with life scale and the emerging construct of life satisfaction. J Posit Psychol. 2008;3(2):137–52.

    Article  Google Scholar 

  56. Watson D, Clark LA, Tellegen A. Development and validation of brief measures of positive and negative affect: the PANAS scales. J Pers Soc Psychol. 1988;54(6):1063.

    Article  CAS  PubMed  Google Scholar 

  57. Thompson ER. Development and validation of an internationally reliable short-form of the positive and negative affect schedule (PANAS). J Cross Cult Psychol. 2007;38(2):227–42.

    Article  Google Scholar 

  58. Gardner B, Abraham C, Lally P, de Bruijn GJ. Towards parsimony in habit measurement: testing the convergent and predictive validity of an automaticity subscale of the Self-Report Habit Index. Int J Behav Nutr Phys Act. 2012;9:1–2.

    Article  Google Scholar 

  59. Verplanken B, Orbell S. Reflections on past behavior: a self-report index of habit strength 1. J Appl Soc Psychol. 2003;33(6):1313–30.

    Article  Google Scholar 

  60. Brooke J. SUS: a quick and dirty usability scale. Usability Evaluation in Industry. 1996.

  61. Glover JD, Tennant SK. Remote areas statistical geography in Australia: notes on the Accessibility/Remoteness Index for Australia (ARIA+ version). The University of Adelaide: Public Health Information Development Unit; 2003.

    Google Scholar 

  62. Bush K, Kivlahan DR, McDonell MB, Fihn SD, Bradley KA, Ambulatory Care Quality Improvement Project (ACQUIP). The AUDIT alcohol consumption questions (AUDIT-C): an effective brief screening test for problem drinking. Archives of internal medicine. 1998;158(16):1789–95.

  63. Smith KJ, McNaughton SA, Gall SL, Blizzard L, Dwyer T, Venn AJ. Takeaway food consumption and its associations with diet quality and abdominal obesity: a cross-sectional study of young adults. Int J Behav Nutr Phys Act. 2009;6:1–3.

    Article  Google Scholar 

  64. Romeo A, Edney S, Plotnikoff R, Curtis R, Ryan J, Sanders I, Crozier A, Maher C. Can smartphone apps increase physical activity? Systematic review and meta-analysis. J Med Internet Res. 2019;21(3):e12053.

    Article  PubMed  PubMed Central  Google Scholar 

  65. Gal R, May AM, van Overmeeren EJ, Simons M, Monninkhof EM. The effect of physical activity interventions comprising wearables and smartphone applications on physical activity: a systematic review and meta-analysis. Sports medicine-open. 2018;4:1–5.

    Article  Google Scholar 

  66. Diggle Peter J, Patrick H, Kung-Yee L, Zeger Scott L. Analysis of longitudinal data. Oxford Statistical Science Series. Oxford: OUP Oxford; 2002.

    Book  Google Scholar 

  67. White IR, Royston P, Wood AM. Multiple imputation using chained equations: issues and guidance for practice. Stat Med. 2011;30(4):377–99.

    Article  PubMed  Google Scholar 

  68. Schoeppe S, Alley S, Rebar AL, Hayman M, Bray NA, Van Lippevelde W, Gnam JP, Bachert P, Direito A, Vandelanotte C. Apps to improve diet, physical activity and sedentary behaviour in children and adolescents: a review of quality, features and behaviour change techniques. Int J Behav Nutr Phys Act. 2017;14:1.

    Article  Google Scholar 

  69. Eysenbach G. The law of attrition. J Med Internet Res. 2005Mar 31;7(1):e402.

    Article  Google Scholar 

  70. Mclaughlin M, Delaney T, Hall A, Byaruhanga J, Mackie P, Grady A, Reilly K, Campbell E, Sutherland R, Wiggers J, Wolfenden L. Associations between digital health intervention engagement, physical activity, and sedentary behavior: systematic review and meta-analysis. J Med Internet Res. 2021;23(2):e23180.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

Not applicable.

Funding

This study is supported by the National Health and Medical Research Council (APP2012704) and the National Heart Foundation of Australia (VG105816). CV is supported by a Future Fellowship from the Australian Research Council (FT210100234). CM is supported by an Investigator Grant from the Medical Research Future Fund (MRF1193862). The funders have had no role in the design of the study and will not have a role in data collection, analysis, interpretation or the writing of manuscripts and dissemination of findings.

Author information

Authors and Affiliations

Authors

Contributions

CV led the development of the study design and development of this manuscript. All authors contributed to developing the study design and protocols. Specifically, DH is the project manager for this study and had considerable input into the practical aspects of designing and implementing this trial. All authors contributed to the development of the intervention concept. All authors (except DH) contributed to obtaining funds for the study. All authors provided critical feedback on draft versions of the manuscript, and they read and approved the final version of the manuscript.

Corresponding author

Correspondence to Corneel Vandelanotte.

Ethics declarations

Ethics approval and consent to participate

The study has received ethics approval from the Human Research Ethics Committee of the Central Queensland University (0000024786). Informed consent to participate will be obtained from all participants.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Vandelanotte, C., Trost, S., Hodgetts, D. et al. MoveMentor—examining the effectiveness of a machine learning and app-based digital assistant to increase physical activity in adults: protocol for a randomised controlled trial. Trials 26, 233 (2025). https://doi.org/10.1186/s13063-025-08926-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13063-025-08926-3

Keywords