Privacy-preserving biometric verification serves as a secure web application for protecting biometric data, which has garnered public attention in recent years. Fully homomorphic encryption (FHE) enables computation on ciphertexts without accessing the secret key, thereby protecting privacy for clients. However, FHE suffers from significant communication overhead and computational costs, which remain the primary bottlenecks in current FHE-based schemes. In this work, we present BioVite, a novel privacy-preserving biometric verification scheme based on techniques from FHE. By adopting Generalized Learning with Errors (GLWE) encryption with compact parameters and optimizing the membership test, BioVite outperforms state-of-the-art FHE-based verification schemes in both runtime and communication size. It requires only around 0.3 ms and 8.5 KB for 512-dimensional biometric templates during verification. In terms of accuracy and precision, \(\textsf{BioVite}\) introduces minimal noise to floating-point similarity computations, and experiments demonstrate that after applying \(\textsf{BioVite}\) , the verification accuracy remains comparable to plaintext verification across various face datasets.