In today’s information age, ensuring comprehensive and effective communication for all individuals, regardless of their abilities, is vital. Sign language (SL) is a natural form of communication used by the deaf as well as the speech-impaired to converse with other individuals and have a proper liaison. This paper aims to use Natural Language Processing (NLP) to implement a system that converts speech to Indian Sign Language (ISL). The paper describes the implementation of a machine translation system that will convert real-time English speech to Indian Sign Language (ISL) automatically. It does this by taking the speech input using the microphone, followed by mapping the words individually with the appropriate gifs or images present in the dataset, and generating the response output as per the ISL grammar. It uses the Python programming language and its supporting libraries, such as EasyGUI (for creating a frontend), PyAudio (for speech input), Sphinx (for speech recognition), and NLTK (for text preprocessing). The software has a wide range of uses, such as in railways, bus stations, and healthcare, to establish proper communication between the mute, deaf, and other people.