Hilbert Basis Activation Function for Neural Network

Artificial neural networks (NNs) have shown remarkable success in a wide range of machine learning tasks. The activation function is a crucial component of NNs, as it introduces non-linearity and enables the network to learn complex representations. In this paper, we propose a novel activation function based on Hilbert basis, a mathematical concept from algebraic geometry. We formulate the Hilbert basis activation function and investigate its properties. We also compare its performance with popular activation functions such as ReLU and sigmoid through experiments on MNIST dataset under LeNet architecture. Our results show that the Hilbert basis activation function can improve the performance of NNs, achieving competitive accuracy and robustness via probability analysis.