Neural Networks have proved their capabilities to model several instances of the modern day. From the last few decades, Machine Intelligence caught great attention, but from the starting of this decade, it rose to great prominence, with the involvement of Artificial Intelligence in a broad (almost every) spectrum of STEM (Science, Technology, Engineering and Management). However, complex the tasks that are being performed by the Neural Networks, the basis behind these architecture to map data points, that are difficult to be separated linearly, to a linearly separable cluster using a combination of neural layers, which are simply a span of linear, (or non-linear) functions, that are better known as Activation Functions. Amongst the most commonly used Activation Functions are Step Functions, Sigmoid, Hyperbolic Tangent, and Rectified Linear Unit (ReLU). Herein, a novel class of Radicalized Hyperbolic Secant Activation Functions (RadSech), is proposed i.e., <InlineEquation ID="IEq1"> <EquationSource Format="TEX">${{\,\textrm{sech}\,}}^{\frac{1}{n}}\left( \cdot \right) $</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <msup> <mrow> <mrow> <mspace width="0.166667em" /> <mtext>sech</mtext> <mspace width="0.166667em" /> </mrow> </mrow> <mfrac> <mn>1</mn> <mi>n</mi> </mfrac> </msup> <mfenced close=")" open="("> <mo>·</mo> </mfenced> </mrow> </math></EquationSource> </InlineEquation>, for <InlineEquation ID="IEq2"> <EquationSource Format="TEX">$n\in \mathbb {Z}^+$</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mi>n</mi> <mo>∈</mo> <msup> <mrow> <mi mathvariant="double-struck">Z</mi> </mrow> <mo>+</mo> </msup> </mrow> </math></EquationSource> </InlineEquation>. The efficacy of the proposed Function has been demonstrated by its abilities to outperform its counterparts, like the Gaussian Activation Function, and Gaussian Error Linear Unit (GeLU). Besides being simpler in action than the counterparts, the robustness of RadSech has been proven on various datasets offered by the <InlineEquation ID="IEq3"> <EquationSource Format="TEX">$\texttt {scikit-learn}$</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mi mathvariant="monospace">scikit</mi> <mo>-</mo> <mi mathvariant="monospace">learn</mi> </mrow> </math></EquationSource> </InlineEquation> library of Python, and across different model complexities.

Radicalized family of hyperbolic secant (RadSech) as an activation function

期刊论文
发表日期：

2025年10月27日

查看全文

Anurag Dutta,
Sanjeev Kumar,
Shanmuga Priya Sivakumar,
Karthik Rajeswaran,
Ramamoorthy Athilingam

摘要

Neural Networks have proved their capabilities to model several instances of the modern day. From the last few decades, Machine Intelligence caught great attention, but from the starting of this decade, it rose to great prominence, with the involvement of Artificial Intelligence in a broad (almost every) spectrum of STEM (Science, Technology, Engineering and Management). However, complex the tasks that are being performed by the Neural Networks, the basis behind these architecture to map data points, that are difficult to be separated linearly, to a linearly separable cluster using a combination of neural layers, which are simply a span of linear, (or non-linear) functions, that are better known as Activation Functions. Amongst the most commonly used Activation Functions are Step Functions, Sigmoid, Hyperbolic Tangent, and Rectified Linear Unit (ReLU). Herein, a novel class of Radicalized Hyperbolic Secant Activation Functions (RadSech), is proposed i.e., ${{\,\textrm{sech}\,}}^{\frac{1}{n}}\left( \cdot \right) $ ${sech}^{\frac{1}{n}} (\cdot)$ , for $n\in \mathbb {Z}^+$ $n \in Z^{+}$ . The efficacy of the proposed Function has been demonstrated by its abilities to outperform its counterparts, like the Gaussian Activation Function, and Gaussian Error Linear Unit (GeLU). Besides being simpler in action than the counterparts, the robustness of RadSech has been proven on various datasets offered by the $\texttt {scikit-learn}$ $scikit - learn$ library of Python, and across different model complexities.