Superior-Order Approximation Functions for Generating Sigmoidal Activation Functions
摘要
Sigmoidal activation functions are frequently used in neural networks for introducing nonlinearity (Popa Electronics 12:24, 2023). Having a sigmoidal shape (S-shape), they have the important advantage of squishing input values into a limited range (usually between 0 and 1 or between − 1 and 1). It exists a multitude of sigmoidal activation functions, such as unipolar sigmoidal activation function, bipolar sigmoidal activation function, Einstein activation functions, hyperbolic tangent activation function or continuous-log sigmoidal activation functions.