<p>A novel approach is proposed that integrates neural networks with function approximation using orthogonal basis functions. A new neural network is proposed in which, instead of feeding raw input data directly into the network, orthogonal polynomials are used to represent the input features. The hidden layer neurons are computed through a tensor product between the weights and the input orthogonal polynomials. Neural networks based on orthogonal polynomials are formulated and proven within a theoretical framework from which the universal approximation theorem is satisfied. Under identical conditions and with the same number of operations, it is demonstrated that the proposed hybrid method outperforms the other in terms of dataset fitting and nonlinear or noisy function approximation. Interpolation as well as extrapolation errors are successfully computed for the Hilbert network and the current neural network. Numerical results are given, and the corresponding figures are depicted.</p>