StepMind: Integrated Foot-Interaction System for Cognitive-Motor Executive Training

Human daily actions rely on the coordinated cognition and movement. This paper presents StepMind, an interactive projection system integrating cognitive-motor training, which enables comprehensive executive function (EF) training through multiple moving targets. StepMind allows users to interact with virtual moving targets in real-time via foot movements, achieving training of EF subcomponents (working memory, inhibitory control and cognitive switching) through multi-target capture tasks, and dynamically adapts to meet diverse user needs. To address the challenges of low sensitivity in foot interaction and multi-target ambiguity in human-computer interaction, we innovatively combine a trivariate Gaussian footfall model with Bayesian intent inference. This framework models the spatial uncertainty of foot placements and applies probabilistic inference to improve target selection accuracy in multi-target scenarios. A 14-day user study with ten participants demonstrated overall EF improvements, while a preliminary deployment in a mental health facility highlighted the system’s potential as a novel, affordable approach to cognitive rehabilitation.