GNN Node Classification Using Koopman Operator Theory on GPU
摘要
Koopman operator theory provides a framework for approximating nonlinear dynamical systems with linear operators. Some research has suggested treating deep neural network (DNN) weights as a dynamical system and utilizing discrete Koopman operators to accelerate network training. We extend former work on this topic in two ways. First, we apply this operation to its most promising use-case of graph neural networks (GNN), demonstrating that the method may be generalized to learning tasks beyond DNNs. Second, we implement Koopman approximations on GPU, significantly accelerating previous CPU-based work. We present an algorithm we call “Patchwork Koopman Approximation” for accelerating GNN training, and we find that performing Koopman approximation can provide a speedup of over three times that of Adam for the same accuracy and loss on the Cora, Citeseer, and PubMed node classification benchmark datasets.