A Social Communication Support Application for Autistic Children Using Computer Vision and Large Language Models

A novel, affordable and accessible software solution that utilizes computer vision tools and Large Language Models (LLMs) to provide communication support to high functioning autistic children during online meetings with the aim of improving their social communication skills is presented. The system displays the remote attendee’s facial expressions as distinct emoticons to facilitate the child’s understanding of non-verbal social cues and suggests appropriate responses on demand based on the conversational context and the detected expressions. A gamification option for practicing facial expression recognition in an engaging manner is also offered. The application serves as a support platform as well as a teaching tool which autistic children can utilize to connect with friends and caregivers to improve their social communication skills. It is being developed in consultation with therapists who work with autistic children to ensure that its design is compatible with the unique needs of the end users. The system is more cost-effective and sensory friendly as compared to similar robotic and virtual reality-based solutions and has the added advantage that the child converses with a real human being instead of a robot or a virtual agent, thus, increasing the likelihood that the social skills learned would be effectively transferred to co-located face-to-face conversations.