The Pi-sigma neural network (PSNN) is a type of high-order feedforward neural network that has product units in the output layer. This gives it a rapid convergence speed and a lot of nonlinear mapping options. This study suggests a gradient descent algorithm with \({\text{L}}_{1}\) regularization and an adaptive momentum term for training PSNN. This paper mainly focuses on two challenging tasks. First, the fact that the general \({\text{L}}_{1}\) regularization is not differentiable at the starting point causes the error function and norm gradient to oscillate during training. This paper's key point is to modify the usual \({\text{L}}_{1}\) regularization term by smoothing it at the origin. This processing yields sparse and efficient neural networks while also providing a theoretical analysis of the algorithm. Second, introduce the adaptive momentum term in the iteration process to further accelerate the network learning speed. In addition, the numerical experiments show that the proposed algorithm eliminates the oscillation and increases the learning rate in computation. We also confirm the algorithm's convergence.
