Optimizing Binary Classification Performance in Neural Networks Through Simulation: A Comparative Study of Activation Functions

Binary classification using Artificial Neural Networks (ANN) is a fundamental problem in machine learning, and the choice of activation functions plays a vital role in determining the performance of the model. This study investigates the performance of various activation functions for binary classification tasks using neural networks across multiple datasets. The results show that ReLU and Tanh, compared to Logistic, consistently excel in accuracy, precision, recall, F1-Score, and AUC-ROC, making them versatile choices for diverse tasks. However, Identity exhibits variable performance, highlighting the need for careful activation function selection based on task specifics. Additionally, the study emphasizes the impact of the dataset size on activation function performance, with ReLU and Tanh offering consistency across varying data volumes. Practitioners can leverage these insights to optimize neural network designs, improving model efficacy in binary classification tasks.