Implementing Cyclical Learning Rates in Deep Learning Models for Data Classification

Neural networks are effectively used in a variety of applications including data mining. The neural network can realize different complex nonlinear functions by making them attractive to identify a system. One of the most important issues of classifying datasets through neural networks is the formation of an ideal network, that consists of many successive steps like set parameters. Perhaps the most prominent parameter is the learning rate. Indeed, choosing an appropriate learning rate value is one of the things that greatly helps to control the overall network performance. In contrast, any inappropriate value for the learning rate negatively affects the classification model and can therefore destabilize the model’s performance and thus seriously deteriorate its quality. This paper presents a new model by adopting a cyclical learning rate instead of using a constant value for training deep neural networks by Multi-Layer Perceptron (MLP) architecture. This model is tested on various real-world datasets; Electricity, NSL- KDD, and four sub-datasets of HuGaDB (HuGaDB-01-01, HuGaDB-05- 12, HuGaDB-13-11, and HuGaDB-14-05). The proposed model achieves an accuracy of, 89.57%, 99.12%, 99.2%, 97.83%, 96.19%, and 99.85% for these datasets respectively. Accordingly, the proposed model outperforms many previous models. As a result, the deep neural network models can be more effective when they adopt an appropriate value for the learning rate.