modSwish: a new activation function for neural network

The activation functions are extremely important to neural networks since they are responsible for learning the abstract characteristics of the data through nonlinear modification. The paper presents a new activation function, which is referred to as modSwish. It is continuously differentiable, unbounded above, bounded below, and non-monotonic. Our results demonstrate that modSwish outperforms ReLU on a number of challenging datasets and neural network models. In the beginning of the experiment, Neural Networks are trained and classified using benchmark data like Diagnostic Wisconsin Breast Cancer and Iris and modSwish achieved 93.57% and 95.56% accuracy, respectively. Secondly, experiments were conducted on five distinct neural network depths that ranged from five to eight layers over MNIST datasets. The modSwish activation function obtained 95.57%, 95.29%, 94.93%, 94.69%, and 95.03% accuracy on 8 layers, 7 layers, 6 layers, 5 thinner layers, and wider 5 layers neural networks respectively. Finally, experiments were conducted on two distinct Convolution Neural Network with two convolution layer and four convolution layers over CIFAR-10 datasets. The modSwish activation function obtained 60.04% and 69.22% accuracy on two convolution layer and four convolution layer model respectively. Statistical feature measurements demonstrate that modSwish has the best mean accuracy, lowest Root Mean squared Error, lowest standard deviation, lowest variance, and lowest Mean squared Error. The study indicated that modSwish has faster convergence compared to ReLU, making it a valuable factor in deep learning. The results of the experiments suggest that modSwish can be a promising substitute for ReLU, leading to better performance in neural network models.