Hierarchical Clustering-Based High-Performance Adjacency Graph Search Algorithm
摘要
Approximate Nearest Neighbor Search (ANNS) based on adjacency graphs is a widely used technique in data retrieval, with significant applications in vector databases and large language models. To address the long index construction time of existing adjacency graph-based ANNS algorithms, this paper proposes a hierarchical clustering-based adjacency graph (HCAG) search algorithm. The method employs hierarchical clustering to obtain a set of centroids, establishes edge connections between centroids and data points, and combines the one-to-many relationship between centroids and data points with the navigability of adjacency graphs to rapidly construct high-performance graph indices. Additionally, a dynamic cluster partitioning strategy is introduced during clustering to ensure more uniform cluster distributions, thereby improving graph quality. Furthermore, a centroid-guided search algorithm is proposed to efficiently traverse query regions at the cluster level, enhancing search performance. Finally, considering the dynamic growth of data scale during incremental graph construction, a dynamic neighbor reservation algorithm is developed to reduce index construction time further. Experimental results on four datasets demonstrate that the proposed algorithm achieves an average 38.12% reduction in index construction time compared to the baseline Hierarchical Navigable Small World (HNSW) method, achieving up to 15-fold improvement in search performance.