SignBuddy: from sign language research to scalable co-created solutions
摘要
This paper presents SignBuddy, the result of ongoing co-created sign language processing research. Most sign language processing research is performed by hearing, non-signing researchers. Even though co-creation efforts have recently increased, technical research still often fails to mention if (and how) co-creation was involved in the research process. SignBuddy is a co-created research tool developed through a partnership between the Flemish Sign Language Centre, a deaf-led organisation, and Ghent University. While respecting elemental concepts of co-creation –– i.e. (i) defining common goals and (ii) building a formal and sustainable relationship between users/consumers and researchers/developers and respecting the five lessons in co-creation –– the platform successfully supported the development of the first fully scalable sign-to-text dictionary search system, built into the Flemish Sign Language–Dutch online dictionary. SignBuddy functions as a crowdsourcing interface for in-the-wild collection of model evaluation data, gathering example queries for quantitative performance analysis and user feedback for qualitative assessment. This human evaluation allows us to shape the application based on the end-users'needs Addressing the need for models that support large dictionaries (over ten thousand signs), we propose a scalable one-shot sign language recognition method and achieve state-of-the-art results. Beyond the co-created application itself, this work provides insights into the co-creation process –– clarifying roles, shared goals, and responsibilities –– and offers conclusions to guide future co-created sign language processing research.