HSP: Efficient Framework for Hamming Similarity Search in PostgreSQL
摘要
Similarity search in Hamming space is a fundamental problem in many applications, such as image retrieval and near-duplicate Web page detection. However, existing open-source implementations for similarity search in Hamming space only provide runtime libraries, leaving issues such as data persistence, data consistency, and data redundancy unaddressed. To solve these problems, we propose HSP, an efficient framework for similarity search in Hamming space in PostgreSQL. Based on this framework, we implement K-ANN search and r-neighbor search, with extensible interfaces for integrating other algorithms. Experiments on several datasets demonstrate the efficiency of our system. We also explore in-database inference, which transforms unstructured data (e.g., image and text) into vectors through representation learning to facilitate applications like image retrieval.