Effects of the Flatness Network Parameter Threshold on the Performance of the Rectified Linear Unit Memristor-Like Activation Function in Deep Learning

In this contribution, we improve of the performance of the Rectified Linear Unit Memristor Like Activation Function with the implication to help training process of CNN without a lot of epochs by computing the best value of the flatness network parameter (p). In this regards the flatness network parameter threshold (p) has been investigated and a good performance of the activation function has been discovered at p = 4.5. We firstly used the MNIST and the CIFAR-10 datasets to trained and test the Alex-Net architecture model of convolutional neural network (CNN) and we showed better performances of Rectified Linear Unit Memristor-like Activation Function compared to those of the literature. We noticed that the performance of Alex-Net also improved and the better performance was recorded when p = 4.5 with 99.50%, 99.25%, 98.81% for training, validation and testing accuracy respectively when using the MNIST. These results open the outcome to reduce the training time of neural networks when this activation function is used.