Packet Performance Predictor Based on Graph Isomorphism Network for Neural Architecture Search

Architecture performance predictor is an important way to evaluate an intermediate neural architecture to improve the efficiency of Neural Architecture Search (NAS). However, it is difficult to learn cluster distribution and isomorphism of architecture representation of the neural network which is very important for the architecture performance prediction. This paper proposes a method named GIN-P3 (Graph Isomorphism Network-based Packet Performance Predictor) to accurately predict the performance of intermediate architectures in NAS. Firstly, we design a clustering-based architecture grouping strategy, which clusters architectures with different characteristics to different groups by using the latent space representation of the architectures based on the variational autoencoder. Secondly, the isomorphism characteristics of each architecture group are learned through Graph Isomorphism Network. Following this, the architecture performance predictor is further constructed and embedded in the NAS based on reinforcement learning through the direct prediction of the architectures sampling by LSTM to improve the efficiency of architecture search. The experiments conducted on the NAS-Bench-101and NAS-Bench-201 show that compared with baseline methods including support vector regression, peephole, gradient-boosted decision trees, and RNAS, GIN-P3 can obtain a higher prediction correlation in performance prediction; further, it can find relatively high-performance neural architecture more quickly.