Integrating Multi-modal Interaction with AR Garment Design for Educational Applications

This paper investigates the strong correlation between interaction methods and scenario requirements within the specialized context of children’s education. It proposes an innovative integration of multimodal interaction techniques to bridge augmented reality (AR), garment design, and educational content. A comprehensive solution based on marker-based visual tracking is developed to support this integration. Key technical challenges such as real-time marker extraction, frame sequence jitter, logo pattern recognition, and texture mapping are addressed. The system allows dynamic combination and presentation of garment card components, enabling interactive learning scenarios. Features such as 360-degree preview and a rotatable mannequin mode are implemented to enhance natural interaction, reduce cost, and improve versatility. Experimental evaluations demonstrate an average processing time of 37 milliseconds per frame and a frame rate of 27 frames per second on mainstream smartphones, meeting real-time application demands. Furthermore, learning effectiveness experiments show that the proposed system significantly enhances children’s learning outcomes, engagement, and overall performance. This work highlights the potential of multimodal AR interactions to transform traditional educational experiences through culturally enriched, visually engaging, and user-centered design.