Accelerating Training of Physics Informed Neural Network for 1D PDEs with Hierarchical Matrices

In this paper, we consider a training of Physics Informed Neural Networks with fully connected neural networks for approximation of solutions of one-dimensional advection-diffusion problem. In this context, the neural network is interpreted as a non-linear function of one spatial variable, approximating the solution scalar field, namely \(y=PINN(x)=A_n \sigma (A_{n-1} ...A_2\sigma (A_1+b1)+b2)+...+b_{n-1})+b_n\) . In the standard PINN approach, the \(A_i\) denotes dense matrices, \(b_i\) denotes bias vectors, and \(\sigma \) is the non-linear activation function (sigmoid in our case). In our paper, we consider a case when \(A_i\) are hierarchical matrices \(A_i=\mathcal{H}_i\) . We assume a structure of our hierarchical matrices approximating the structure of finite difference matrices employed to solve analogous PDEs. In this sense, we propose a hierarchical neural network for training and approximation of PDEs using the PINN method. We verify our method on the example of a one-dimensional advection-diffusion problem.