Scholarlink

This research introduces an innovative social robotic system for emotion education therapy in children with autism spectrum disorder. Integrating a NAO robot with dual interfacesone displaying emotional stimuli for children and another providing facilitator oversightour approach implemented a progressive five-session therapeutic framework. Each session targets distinct emotions through increasingly complex multimodal exchanges, from basic dialogues to comprehensive social scenarios involving verbal, facial, bodily, and contextual emotional cues. The system harnesses advanced technologies including ChatGPT/Whisper for adaptive conversation, DeepFace for affective state recognition, and MediaPipe for postural analysis. Our holistic design incrementally increases interactional complexity across five structured activities: introductory conversation, facial emotion deciphering/expression, bodily emotion conveyance, robot-narrated stories with integrated emotional gesture demonstrations, and a concluding musical interlude. Initial findings substantiate the potential of robotic interventions to provide structured emotional literacy development for children with ASD while maintaining essential human guidance via an accessible control mechanism. This work contributes to the growing field of socially assistive robotics by demonstrating how progressive multimodal interactions can support specialized emotional learning interventions.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

A Progressive Multimodal Robot System for Emotional Learning in Autistic Children

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Yiyi Wu,
  • Maninderjit Kaur,
  • Fengpei Yuan

摘要

This research introduces an innovative social robotic system for emotion education therapy in children with autism spectrum disorder. Integrating a NAO robot with dual interfacesone displaying emotional stimuli for children and another providing facilitator oversightour approach implemented a progressive five-session therapeutic framework. Each session targets distinct emotions through increasingly complex multimodal exchanges, from basic dialogues to comprehensive social scenarios involving verbal, facial, bodily, and contextual emotional cues. The system harnesses advanced technologies including ChatGPT/Whisper for adaptive conversation, DeepFace for affective state recognition, and MediaPipe for postural analysis. Our holistic design incrementally increases interactional complexity across five structured activities: introductory conversation, facial emotion deciphering/expression, bodily emotion conveyance, robot-narrated stories with integrated emotional gesture demonstrations, and a concluding musical interlude. Initial findings substantiate the potential of robotic interventions to provide structured emotional literacy development for children with ASD while maintaining essential human guidance via an accessible control mechanism. This work contributes to the growing field of socially assistive robotics by demonstrating how progressive multimodal interactions can support specialized emotional learning interventions.