Non-revisiting Stochastic Search for Automatic Graph Learning

In recent years, automatic graph learning (AutoGL) has been widely concerned by academia and industry because it can significantly reduce the threshold and labor cost of graph learning. It has shown powerful functions in hyper-parameter optimization, model selection, graph neural architecture search, and feature engineering. With the development of the network structure, the time and computing resources consumed by the AutoGL process are increasing. AutoGL can be viewed as a bilevel optimization problem encompassing inner and outer optimization. The inner optimization focuses on optimizing the model parameters through techniques like stochastic gradient descent, aiming to minimize the loss function and enhance model performance. On the other hand, the outer optimization aims to identify the best configuration settings for hyperparameters and neural network structures. To address the computational cost associated with heuristic algorithms in bilevel optimization for AutoGL, the non-revisiting idea is proposed to keep a record of all previously evaluated individuals to avoid redundant search. Experiments on multiple search algorithms demonstrate that non-revisiting can improve the time performance of AutoGL. The time performance of the search algorithm with non-revisiting is improved by 40 \(\%\) to 80 \(\%\) compared with that without non-revisiting under the condition that the experimental accuracy remains unchanged.