Approximate nearest neighbor search is a critical component in large-scale high-dimensional retrieval systems. This paper addresses the 2025 SISAP Indexing Challenge task 1, which requires searching 23 million 384-dimensional vectors under tight memory and disk constraints, with an average recall threshold of 0.70. To meet these constraints, we introduce two optimizations to the DiskANN pipeline. First, we apply PCA-based dimensionality reduction to ensure memory usage stays within bounds. Second, we refine the second-assignment mechanism by sorting points based on distance and assigning only the closer half to the secondary shard. Our approach achieves approximately 0.80 recall while adhering to the challenge’s resource and performance limitations.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Memory-Constrained DiskANN: Efficient Approximate Nearest Neighbor Search Under Resource Constraints

  • Yuhang Lou,
  • Linyun Ma,
  • Kun Luo,
  • Yan Ruan,
  • Huijia Wu,
  • Minhua Lu,
  • Rui Mao

摘要

Approximate nearest neighbor search is a critical component in large-scale high-dimensional retrieval systems. This paper addresses the 2025 SISAP Indexing Challenge task 1, which requires searching 23 million 384-dimensional vectors under tight memory and disk constraints, with an average recall threshold of 0.70. To meet these constraints, we introduce two optimizations to the DiskANN pipeline. First, we apply PCA-based dimensionality reduction to ensure memory usage stays within bounds. Second, we refine the second-assignment mechanism by sorting points based on distance and assigning only the closer half to the secondary shard. Our approach achieves approximately 0.80 recall while adhering to the challenge’s resource and performance limitations.