Comparing Incremental Learning Approaches for a Growing Sign Language Dictionary

Machine Learning-based Sign Language Dictionaries recognize a sign performed in front of a camera and return the most probable written language word. Due to the scarce number and variety of datasets for sign languages, these dictionaries need an incremental approach to include new signs each time a new dataset is available. Current sign language recognition models used in these dictionaries are trained for a fixed number of classes. For this reason, our work systematically compares three incremental learning approaches in a skeleton and transformer-based sign language recognition model to train up to 60 classes or Peruvian Sign Language (LSP) signs. In addition, we also evaluate two distinct incremental groups: only taking new classes and, taking new instance of old and new classes. This last incremental group is considerably less explored compared to the incremental approach involving only new classes at each step. We found that a method inspired by distillation loss outperforms others in most scenarios.