Don’t Let Your Remotes Flop! Potential Ways to Incentivize and Increase Study Participants’ Use of Edtech

Studies that require participants to use educational apps or other technologies have often struggled with fidelity of implementation and even user uptake [e.g., 1]. That is, it is often unclear how much the participants have used the designated technologies at home or in the classroom when only self-reports are used to assess participants’ usage. In the event that the information can be objectively obtained using process data, the participants’ actual usage tends to be lower than what has been reported or observed via synchronous sessions [2]. Because remote studies on the impacts of technological innovations or interventions can often depend upon how much study participants actually use them independently, it is important to have strategies on-hand to increase the at-home use of these technologies. Our poster presents two years of data from a study focused on an educational app meant to improve children’s early literacy through modeling dialogic reading practices with an AI-powered conversational agent. The app features a virtual rabbit-agent who asks questions to prompt caregiver-child interactions as the caregiver reads a physical book aloud with their child. In the first year, focused on usability, we discovered there was sparse usage of the app, with 6 of the 20 participants procrastinating until the day before their post-test session to use it. We implemented four key changes–device, reading diary, text reminders, and compensation structure–to increase and incentivize participant usage of the app during the study’s second-year efficacy phase. Although we are unable to distinguish the unique contribution of the four changes implemented to increase user engagement with the app, clustering of the process data patterns suggests a clear increase in the app’s usage during the second year of the study. Our findings can be useful to researchers and practitioners facing similar implementation challenges.