Designing a Serious Game with Natural Interaction to Support the Learning of Chinese Sign Language and Deaf Social Etiquette

Sign language is an important communication means among deaf community or between deaf and non-deaf people. This paper presents a serious game (called “Finger Burger”) that utilizes natural interaction, situated learning, animation and narratives aims to teach both deaf and non-deaf individuals Chinese Sign Language (CSL) and deaf social etiquette. In order to explore the role of natural interaction on CSL learning gains and game experiences, we conducted a preliminary evaluation with 12 young adults who used both our natural interaction (NI) game and a similar version but with a mouse-click interaction (MCI). We mainly used quantitative methods to analyze the data, supplemented by qualitative results. The results demonstrated our games could significantly improve users’ learning gains on CSL and deaf social etiquette. Although the MCI version exhibited slightly better performance in CSL learning, the NI version excelled in executing complex sign tasks. The NI version also yielded superior learning gains in social etiquette. Participants in the NI reported a generally more positive game experience compared to those in the MCI condition. We discuss the affordances and challenges in designing natural interaction for sign language learning.