With the increasing prevalence of voice assistant services (VAS), ensuring system security and user privacy has become a significant challenge. Preliminary analysis of existing authentication mechanisms reveals shortcomings, particularly in multi-user settings and the reliance on additional devices. To address this, we propose a novel approach that embeds users’ biometric templates into the neural network model of voice assistants for identity authentication. Leveraging the robust sound processing capabilities of CNNs, this method employs watermark technology within the model for user identity verification. Experimental results demonstrate that this method effectively verifies user identities while the impact on the original model’s performance can be negligible. Evaluation continuation with 10 participants and 300 different voice commands revealed an overall accuracy of 99.01% and an equal error rate (EER) of 1.25%.