Listen to Air Canvas
摘要
Air writing, which involves writing characters in mid-air using hand gestures, has gained traction as a touchless and accessible method of interaction, particularly useful in smart environments, education, and for individuals with speech or mobility impairments. While significant advancements have been made in gesture recognition, text recognition, and sign language translation using deep learning models like CNNs, RNNs, and LSTMs, limited research has fully integrated air writing with audio conversion. Studies have explored various recognition methods, from Myo armbands and sEMG signals to video-based systems and OCR techniques, achieving high accuracies in character and word recognition. Research has also addressed trajectory tracking, stroke analysis, and pre-trained models for text recognition. However, most systems stop at text conversion, with only a few extending to synthesized speech using tools like gTTS or applying ML to translate gestures into audio feedback. Hardware dependence and incomplete gesture-to-audio pipelines remain challenges. This study aims to overcome these limitations by developing a seamless system that captures air writing strokes, converts them into visual frames and text, and finally into audio, thus offering a more accessible and interactive experience. The novelty lies in its end-to-end integration without heavy hardware dependence, promoting inclusivity and ease of use for broader applications.