Neural acceleration of incomplete factorization preconditioning

The solution of a sparse system of linear equations is ubiquitous in scientific applications. Iterative methods, such as the preconditioned conjugate gradient (PCG) method and the generalized minimal residuals (GMRES) method, are normally chosen over direct methods due to memory and computational complexity constraints. However, the efficiency of these methods depends largely on the preconditioner utilized. The development of a preconditioner normally requires some insight into the sparse linear system and the desired trade-off between generating the preconditioner and the reduction in the number of iterations. Incomplete factorization is a popular black box method to generate these preconditioners. However, it may fail for several reasons, including numerical issues that require searching for adequate scaling, shifting, and fill-in while utilizing a difficult-to-parallelize algorithm. With a move toward heterogeneous computing, many sparse applications find GPUs that are optimized for dense tensor applications like training neural networks being underutilized. In this work, we demonstrate that a simple artificial neural network trained either at compile time or in parallel to the running application on a GPU can provide an incomplete \(LL^{T}\) $L{L}^T$ factorization that can be used as a preconditioner. This generated preconditioner is as good as or better than the ones found using multiple preconditioning techniques such as scaling and shifting in terms of reduction in number of iterations. Moreover, the generated method also works and never fails to produce a preconditioner that does not reduce the iteration count.