Communication barriers between hearing-impaired individuals and the general population remain a significant challenge in inclusive societies. This paper presents an integrated real-time bidirectional communication system designed to bridge this gap by translating Indian Sign Language (ISL) gestures into speech and converting spoken English into corresponding ISL visual cues. The proposed system features an intuitive interface developed with Python’s Tkinter library, combining ease of use with accessibility, using deep learning and computer vision for gesture recognition, and natural language processing for voice input. The gesture-to-voice module uses a webcam to capture live video streams, from which keypoints are extracted using MediaPipe’s holistic model. A proposed customized pre-trained CNN-LSTM-based deep learning model classifies sequences of keypoints into ISL gestures, which are then vocalized using a text-to-speech engine. Conversely, the speech-to-sign module employs Google’s speech recognition API to transcribe spoken English, which is then translated into ISL via pre-stored animated GIFs or letter-based visual cues. This dual-mode interface empowers both hearing-impaired and non-impaired users to engage in natural interaction, with real-time performance and modular scalability. Our system is lightweight, offline-capable for gesture recognition, and adaptable to different vocabularies and regional sign variations, thereby offering a practical assistive technology solution for inclusive communication.