Computer vision, audio processing, natural language processing, and many other fields widely use deep learning. In deep learning algorithms, hyperparameters have a various impact on model processing. Unfortunately, because of the nature of these deep neural models, it is not easy to define the hyperparameters of complicated models. This work implemented long short-term memory with sparrow search algorithm (LSTM-SSA) model that is used for hyperparameter optimization and a hybrid model of BERT and GRU for Hindi hate speech detection. The present models employ neural network to identify and categorize occurrences of hate speech as hate or non-hate. Experiments with basic and complicated network situations verify the proposed strategy, where human intervention has major impact on performance of model. With the help of fitness function of sparrow search algorithm, the impact of human intervention is reduced as it computes the distance, position, and fitness of the parameter to target so that weights do not affect the performance of model much. The G-BERT model seeks to improve hate speech detection by utilizing the sequential information collection skills of GRU and the contextual understanding of BERT. The results show that G-BERT model performs better than LSTM-SSA model by achieving accuracy of 91% where LSTM-SSA model achieved accuracy of 87% in detecting Hindi hate speech.