Mobile Application for Learning Polish Sign Language

The increasing awareness and popularization of sign language in many countries have contributed to the development of mobile educational applications. This also applies to Polish Sign Language, used by the Deaf community in Poland. However, among the available solutions, there is a need for one which would engage the user in independent signing, i.e., forming the necessary finger motor skills and dexterity. To address this challenge, the paper focuses on developing a system for recognizing Polish Sign Language symbols that could work inside a mobile application. The publicly available MediaPipe library is used to determine the hand’s characteristic points and their coordinates. The detected points are then normalized and stored. Three machine learning techniques are verified at the character recognition stage: support vector machine, k-nearest neighbors, and random forest. Each was taught on the coordinate values obtained from a set containing 29,945 images of hands, divided into 29 classes (considering selected letters, supporting characters, and numbers). The obtained accuracies of all the methods are equal to about \(98\%\) .