The Pipeline of Sign Language Machine Translation

This chapter provides a comprehensive overview of the research conducted in the dynamic field of Sign Language Machine Translation (SLMT) over the years. We dissect the SLMT process into a three-component pipeline comprising: (i) sign language recognition, which involves extracting information from input videos containing signed utterances and processing it into a suitable format for any downstream task; (ii) machine translation, the task of converting automatically, through the use of a computer system, an input sequence in one language (the source) into another language (the target) while preserving its meaning; and (iii) sign language synthesis (SLS), which entails generating signed utterances in the target sign language. In addition to the technical aspects, this chapter emphasizes ethical considerations and offers some guidelines that we strongly believe researchers must adhere to when conducting research in the domain of SLMT. Ensuring ethical practices is of the utmost importance given the nature of this research, which closely involves deaf communities, their languages and their cultures. While discussing various contributions in fields related to SLMT, this chapter aims to provide a broad background to support a comprehensive high-level understanding of that work, without delving into intricate details. It also identifies the interconnections among these fields and highlights the corresponding challenges faced in advancing SLMT research.