Similarity search in Hamming space is a fundamental problem in many applications, such as image retrieval and near-duplicate Web page detection. However, existing open-source implementations for similarity search in Hamming space only provide runtime libraries, leaving issues such as data persistence, data consistency, and data redundancy unaddressed. To solve these problems, we propose HSP, an efficient framework for similarity search in Hamming space in PostgreSQL. Based on this framework, we implement K-ANN search and r-neighbor search, with extensible interfaces for integrating other algorithms. Experiments on several datasets demonstrate the efficiency of our system. We also explore in-database inference, which transforms unstructured data (e.g., image and text) into vectors through representation learning to facilitate applications like image retrieval.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

HSP: Efficient Framework for Hamming Similarity Search in PostgreSQL

  • Shengsong Liu,
  • Jiangfeng Xiao,
  • Yaoshu Wang,
  • Hua Yan,
  • Yan Ji,
  • Rui Mao,
  • Jianbin Qin

摘要

Similarity search in Hamming space is a fundamental problem in many applications, such as image retrieval and near-duplicate Web page detection. However, existing open-source implementations for similarity search in Hamming space only provide runtime libraries, leaving issues such as data persistence, data consistency, and data redundancy unaddressed. To solve these problems, we propose HSP, an efficient framework for similarity search in Hamming space in PostgreSQL. Based on this framework, we implement K-ANN search and r-neighbor search, with extensible interfaces for integrating other algorithms. Experiments on several datasets demonstrate the efficiency of our system. We also explore in-database inference, which transforms unstructured data (e.g., image and text) into vectors through representation learning to facilitate applications like image retrieval.