Automatic Speech Recognition (ASR) is an emerging technology in today’s digital world, allowing for seamless human–computer interaction via voice commands, transcription services, and other features. This work presents an end-to-end model capable of transcribing the real-time captured speech into text. A Customized Bi-directional Gated Recurrent Unit (Bi-GRU) with convolutional neural network (CNN) along with the technique of layer normalization. The model incorporates convolutional layers to robustly extract features from raw audio signals, combined with layer normalization and Bi-GRUs to capture long-range temporal dependencies and contextual information from both past and future frames. Significant improvements were observed over five training epochs. The training and validation loss have been reduced from 311.9084 to 70.7906 and 303.5839 to 46.6886, respectively. The dataset provides comprehensive metadata in transcript file, assisting an extensive number of research initiatives in speech synthesis and synthetic speech detection. The audio clip encoded as a single-channel 16-bit PCM.wav file at a sample rate of 22,050 Hz. The experiment was done to design more accurate and reliable speech recognition systems and highlight the effect of layer normalization in reducing the CTC loss.