Real-Time Arabic Speech Recognition from Lips Movement Based on Deep Learning
摘要
The ability to hear and easily communicate with others is a blessing that some people with hearing impairments lose. Lip-reading may be a solution to this problem. This paper proposes a new lip-reading technique for recognizing Arabic speech. Due to the lack of a public Arabic dataset, a new dataset was created comprising over 2480 videos. The proposed technique includes several stages: face detection, lip alignment, lip region segmentation, and recognition. The initial stage utilizes mediapipe for face detection, followed by proposed lip alignment. Then, accurate delineation of the lip region is performed using a cropping method that removes extraneous elements such as moustaches. Additionally, a new technique selects sixteen different frames and eliminates redundant frames, fusing them into one image. Finally, Modified VGG16 is used to recognize image labels. The proposal's performance was remarkable, with accuracy exceeding 97.45% for words and 98.03% for phrases, surpassing previous Arabic recognition systems.