HEMQ: Towards More Accurate Post-training Mixed-Precision Quantization of Neural Networks
摘要
Model quantization reduces latency, improves speed, and saves energy by making models smaller and less demanding to run. Post-Training Quantization (PTQ) is faster than Quantization-Aware Training (QAT) methods but often hurts accuracy when using very low precision. Mixed-precision quantization assigns different precision levels to different layers to balance accuracy and efficiency, but finding the best combination is difficult due to too many possible configurations and high evaluation costs. This paper introduces HEMQ (Hybrid Evolutionary Mixed-Precision Quantization), a new approach for PTQ that uses a fast error measurement method and combines genetic algorithms with differential evolution to efficiently search for optimal precision settings. It also uses a two-stage optimization process to improve accuracy. Experiments show that HEMQ outperforms existing methods across various tasks and network types, providing a practical solution for running high-performance models on devices with limited resources.