Enhancing vision–language model calibration via rotation-enhanced orthogonality-constrained prompt tuning

Test-time prompt tuning (TPT) has emerged as a promising approach to enhance the adaptability of vision–language models (VLMs) without requiring additional labeled data, making it suitable for open-set or out-of-domain scenarios. However, TPT methods often suffer from poor calibration, compromising model reliability. This paper proposes a rotation-enhanced orthogonality-constrained test-time prompt tuning (Ro-TPT) method to address this issue. By incorporating a random rotation mechanism into the feature space based on an orthogonality constraint framework, Ro-TPT dynamically rotates text features to optimize directional alignment. Simultaneously, a new reward function is introduced to search for the optimal rotation space, achieving better calibration. Extensive experiments on multiple benchmark datasets demonstrate that Ro-TPT significantly improves calibration performance while maintaining prediction accuracy, outperforming existing methods. Here, we show that the proposed Ro-TPT reduces the expected calibration error by up to 36% on average compared to the traditional TPT and O-TPT methods. The relevant code is published on https://github.com/Kzhou05/Ro-TPT.