<p>The recently proposed concurrent ordered index (XIndex-R) based on Learned Indexes (LIs) demonstrates significant improvements in write scalability and query performance for large-scale indexing. However, the performance of XIndex-R remains constrained by inadequate consideration of input key distributions, particularly across diverse data domains. In this paper, we present the Transformation-Partitioned Learned Index (TPLI), a distribution-aware preprocessing framework that enhances the state-of-the-art concurrent learned index XIndex-R for multi-domain key-value stores. It improves indexing efficiency while preserving concurrency by tailoring its architecture to key types. For numeric keys, to enable better approximation of the linear Cumulative Distribution Function (CDF), a numerical Normalizing Flow (NF) is employed to transform skewed numeric key distributions into near-uniform distributions. For string keys, to reduce the operation overhead and improve model specialization, the K-means clustering approach with the elbow method is utilized to partition keys into disjoint subsets, each modeled by a dedicated learned model. Experimental evaluations on real-world numeric and string datasets demonstrate that TPLI outperforms six state-of-the-art methods, achieving superior throughput for both query and write operations.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

TPLI: Transformation-partitioned learned index for multi-domain key-value stores

  • Meng Zeng,
  • Bin Ning,
  • Qiong Gu,
  • Chunyang Hu,
  • Qiaozhi Hua

摘要

The recently proposed concurrent ordered index (XIndex-R) based on Learned Indexes (LIs) demonstrates significant improvements in write scalability and query performance for large-scale indexing. However, the performance of XIndex-R remains constrained by inadequate consideration of input key distributions, particularly across diverse data domains. In this paper, we present the Transformation-Partitioned Learned Index (TPLI), a distribution-aware preprocessing framework that enhances the state-of-the-art concurrent learned index XIndex-R for multi-domain key-value stores. It improves indexing efficiency while preserving concurrency by tailoring its architecture to key types. For numeric keys, to enable better approximation of the linear Cumulative Distribution Function (CDF), a numerical Normalizing Flow (NF) is employed to transform skewed numeric key distributions into near-uniform distributions. For string keys, to reduce the operation overhead and improve model specialization, the K-means clustering approach with the elbow method is utilized to partition keys into disjoint subsets, each modeled by a dedicated learned model. Experimental evaluations on real-world numeric and string datasets demonstrate that TPLI outperforms six state-of-the-art methods, achieving superior throughput for both query and write operations.