Privacy-preserving facial verification aims to authenticate an individual’s identity without exposing their facial characteristics, thereby protecting user privacy. While current verification schemes based on fully homomorphic encryption (FHE) are practical, they often suffer from inefficiencies, such as requiring multiple communication rounds, relying on trusted third parties, or involving large-size ciphertexts. In this work, our objective is to craft a more efficient and compact FHE-based facial verification scheme that operates within a single communication round in the two-party setting. By leveraging preprocessed hints stored on the client side, our new scheme requires less computation cost, achieving a total computation time of approximately 1.35 ms (20 \(\times \) faster than the state-of-the-art) and a communication overhead around 4 KB (a significant reduction compared to over 128 KB in previous schemes) for 512-dimensional facial templates. Experimental results also demonstrate that the accuracy of our privacy-preserving verification closely matches the verification in cleartext, ensuring a high practicability and usability.