An intelligent and ergonomic wearable gesture driven mathematics learning system for persons with disabilities in inclusive education
摘要
Traditional teaching techniques, with their heavy reliance on oral instruction and written practice, create serious barriers for persons with speech, mobility, visual, or learning disabilities. This work introduces a multisensory, gesture-based learning device to provide an inclusive and interactive mathematics platform. The main input modality is through gestures. The users wear a motion sensor to perform mid-air gestures, representing digits from zero to nine and mathematical operators like addition and subtraction to answer computer-generated problems. A customized, lightweight deep learning model recognizes gestures by interpreting their spatial and temporal gesture-based features. The model is optimized for efficiency by making the system compact and minimizing delay while preserving the accuracy required for correct recognition of intended numbers and symbols. The recognized gestures are fed into a large language model that generates relevant math problems and provides real-time audio feedback. The system automatically adjusts problem difficulty based on user performance, creating a personalized learning experience. The system was evaluated with twenty persons with disabilities, representing the target population, demonstrating strong effectiveness, usability, and accessibility. The proposed gesture-based approach demonstrates strong promise in empowering persons with disabilities by closing gaps in accessibility and encouraging engagement through adaptive, multisensory learning.