Inaccurate modeling of affective states and engagement in children with Autism Spectrum Disorder (ASD) remains a critical barrier to adaptive and therapeutic platforms. This work introduces an interpretative Affective Knowledge Graph (AKG), which integrates behavior, physiology, and affective modalities to predict affective states in a way that is understandable, accurate, and efficient. An aggregate multimodal dataset is developed by integrating representative attributes such as engagement rate, galvanic skin responses, temperature, and valence-arousal values from behavior, affective, and clinical domains. Weakly supervised emotion annotations are obtained from self-supervised learning signals, with weights learned on confident instances with non-negativity and sparsity constraints. Five-fold cross-validation experiments revealed reliable prediction accuracy and sound reasoning behavior, with interpretative AKG analysis supporting complementary contributions from different modalities that are consistent with existing models of affect in ASD. The AKG provides a constructive, interpretive base for developing adaptive learning platforms, where interpretability of the prediction, that is, the reasoning behind predicting a particular affective state, is as important as the accuracy of that prediction.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

An Explainable Affective Knowledge Graph Framework for Multimodal Emotion and Engagement Modeling in Autism Spectrum Disorder

  • Nisrine El Ayat,
  • Mohammed Boutalline,
  • Adil Tannouche,
  • Chaimae Ouazri

摘要

Inaccurate modeling of affective states and engagement in children with Autism Spectrum Disorder (ASD) remains a critical barrier to adaptive and therapeutic platforms. This work introduces an interpretative Affective Knowledge Graph (AKG), which integrates behavior, physiology, and affective modalities to predict affective states in a way that is understandable, accurate, and efficient. An aggregate multimodal dataset is developed by integrating representative attributes such as engagement rate, galvanic skin responses, temperature, and valence-arousal values from behavior, affective, and clinical domains. Weakly supervised emotion annotations are obtained from self-supervised learning signals, with weights learned on confident instances with non-negativity and sparsity constraints. Five-fold cross-validation experiments revealed reliable prediction accuracy and sound reasoning behavior, with interpretative AKG analysis supporting complementary contributions from different modalities that are consistent with existing models of affect in ASD. The AKG provides a constructive, interpretive base for developing adaptive learning platforms, where interpretability of the prediction, that is, the reasoning behind predicting a particular affective state, is as important as the accuracy of that prediction.