Although speech translation software facilitates communication between speakers of different languages, many of the systems in use today struggle to translate idioms and exquisite meanings that are crucial for effective communication. The goal of this project is to develop a novel speech translation system designed especially for translating English to Tamil, Telugu, and Hindi. To improve translation accuracy, particularly with regard to idioms and context, the study employs advanced algorithms and cutting-edge technologies. Techniques like domain-specific training and semantic modeling will be used by the system to assist it adjust to specialized domains like technical subjects. One significant advancement is the creation of discrete speech units using self-supervised learning, which enables the system to comprehend spoken language details without requiring a large amount of labeled data. In order to ensure that translations sound fluid and natural, the study also makes use of contemporary neural network models, such as improved transformers, which are adept at managing complex linguistic patterns. The system will also have two new features: a context-aware module that adapts translations to fit particular fields and their particular words, and an idiom recognition tool that locates and accurately translates idiomatic phrases from English to Telugu, Hindi, or Tamil. The goal of these improvements is to improve accuracy in specific areas while maintaining idiomatic meanings in order to address the issues with the current systems.