Auto-tuning Compiler Flags with Pretrained Language Models and Surrogate-guided Search

Compiler optimization is a crucial step in software development, aiming to enhance program performance by selecting the most suitable optimization flags. However, the vast search space of compiler flags, often exceeding hundreds, poses significant challenges for developers. This paper introduces a novel framework that leverages pretrained language models (PLMs) to extract rich code representations for compiler auto-tuning. A lightweight surrogate model predicts program performance based on these representations and optimization flag settings, while GASM (Genetic Algorithm via Surrogate Model) efficiently explores the configuration space through adaptive fine-tuning of the surrogate model. Experiments on PolyBench and CBench demonstrate that our approach achieves up to 2x faster optimization with comparable or better speedups than state-of-the-art methods like RIO, OpenTuner, and BOCA. The surrogate model achieves a 6% Mean Absolute Percentage Error, ensuring accurate performance predictions. This work establishes a scalable and efficient foundation for future research in automated compiler optimization.