Speech is a mode for humans to express their emotions. We recognize its ease of use while opting to other means of communication like text, where we frequently utilize emotion to convey our feelings. Emotion detection has become essential in today's world of communication, user communicate through various digital means through audio and video conferences. The speech in fear, sadness, and joy have a higher and wide range in pitch, whereas others have a low range in pitch. In existing systems, SVM and MLP techniques predict speech emotion. Speech-based emotion recognition (SER) system is used to recognize different emotions using convolution neural network (CNN) and Recurrent neural network (RNN) classifiers. Librosa package is one of the widely used speech recognition package in Python is adopted with the proposed model, and evaluated through CREMA dataset that consists of audio data of different emotions, like surprise, anger, sadness, and fear. The output makes the calculations based on the fundamental frequency of each speech frame to map the raw speech data straight to a textured image. The textured images produced by the conversion can be categorized using deep neural network models for emotion recognition.