Secure speech retrieval method using deep hashing and CKKS fully homomorphic encryption

The development of deep learning technology makes speech retrieval and recognition more accurate and efficient. Meanwhile, the privacy leakage problem of speech data is becoming increasingly prominent, but the emergence of fully homomorphic encryption (FHE) technology can alleviate the concerns about privacy information. In order to protect the privacy of speech data and deep binary hash codes, and realize the privacy-preserving similarity calculation, a secure speech retrieval method using deep hashing and CKKS (Cheon-Kim-Kim-Song) FHE was proposed. Firstly, a speech CKKS FHE scheme is designed to encrypt the original speech data. Then, the spectrogram image features of the original speech data are extracted as the input of triplet convolutional neural network (Tri-CNN) to generate efficient and compact deep binary hash codes, which are encrypted and uploaded to the cloud together with the encrypted speech data. When retrieving, the deep binary hash codes of the querying speech is extracted, encrypted and sent to the cloud server as a search trapdoor, and the security similarity is calculated with the index sequence in the secure index table. The experimental results show that the mean average precision of the proposed method in the TIMIT and THCHS-30 data sets is more than 93%, with a loss of about 2% compared with the plaintext domain, but with higher security.