Linguistic Processing for Sign Language Translation

Natural Language Processing (NLP) has substantially advanced in recent years with the establishment of deep learning paradigms in areas such as machine translation and text generation. Considerable quantitative performance gains have been achieved in machine translation for spoken languages. However where machine translation for sign languages is concerned, advances are much more modest not only because of the unavailability of annotated datasets but also because of sign languages’ inherent non-linearity and production in the visual-gestural modality. Since sign languages have a structure of comparable complexity to spoken and written language and perform a similar range of functions, sign language processing tools should be able to uncover these structures from a multimodal stream of information to support different language processing applications. In particular, where machine translation between pairs of spoken and signed languages is of concern, linguistic processing of the input could contribute to alleviate dataset scarceness. In this chapter, we overview what types of linguistic information (and processing) could be used to contribute to machine translation results between spoken and signed languages, presenting both rule-based and machine learning approaches.