In past decades, speech synthesis methods based on Deep Learning (DL) has been used to model the raw speech waveform and successfully generated natural-sounding waveforms. However, the autoregressive, non-autoregressive, flow-based, and GAN-based models that produced acoustic features using intermediate prediction of Mel spectrogram and subsequently used vocoders to generate the raw waveforms. Autoregressive and flow-based architecture requires a million of steps to achieve a realistic waveform generation capability, which is resource-intensive. In this paper, we introduce Faster Convergence HiFi-GAN (FCHiFi-GAN) exploit batch-wise normalization method to identify and capture data distribution effectively. There is a less number of architectures are capable of fast training. The proposed architecture got the real-like speech generation with only 600K steps, whether existing state-of-the-art architectures need 2.5M (16 batch size) and 580k (24 batch size) steps, respectively, for HiFi-GAN and WaveGlow. However, the number of steps varies depending on the dataset and total number of parameters. We trained the HiFi-GAN with the same setting for evaluation and got noticeably good results over the baseline architecture. To evaluate the performance of the FCHiFi-GAN, we measured and analysed generated samples with existing architecture using subjective (Mean Opinion Score (MOS)) and objective (Mel Cepstral Distance (MCD)) measures, Perceptual Evaluation of Speech Quality (PESQ), Signal-to-Noise Ratio (SNR), and Modulation Spectra Distance (MSD). FCHiFi-GAN generated samples achieve 4.42 MOS (+0.06 than baseline architecture) while reducing the required computational cost to generate high-quality samples.