Real-Time Continuous Tamil Dialect Speech Recognition and Summarization

The proposed research work addresses the challenge of developing a near real-time Tamil dialect-based transcription system. The primary aim is to improve automatic speech recognition (ASR) for Tamil dialects, focusing on minimizing latency and enhancing accuracy and providing an open-source solution to tackle the problem of recognizing accented speech. In this regard, we introduce a novel Tamil dialect-based speech and text corpus to fine-tune a whisper model, specifically the whisper small architecture, to generate accurate transcripts. To overcome resource constraints, we employed a parameter-efficient fine-tuning technique called low ranking adapters. The fine-tuned models achieved a 61% word error rate for dialectal speech, a significant improvement over the pretrained models which achieved over 80%. Additionally, we explored dialect classification using the whisper encoder with a classification head, achieving near-perfect accuracy of 97.1%. Furthermore, we demonstrated a practical application of the ASR system by integrating it with a summarization tool to generate intermediate summaries for continuous speech. This research work contributes to advancing ASR technology for Tamil dialects, with implications for improved communication and accessibility in diverse linguistic contexts.