Recently, advancements in storage architectures and query optimization techniques have significantly improved the performance of database, driving the evolution of database systems. Moreover, the demand for database systems to quickly index and retrieve large volumes of data has increased substantially. The log-structured merge tree (LSM-tree), recognized for its excellent write performance, is particularly well-suited for NoSQL database due to its capability to handle large datasets and facilitate high-frequency writes and updates to indexes and data. However, the multi-level storage structure of LSM-tree necessitates searches in multiple sorted string tables during read operations, resulting in high disk I/O overhead and increased query times, which decreases the query efficiency of database systems. To address this challenge, we propose a machine learning-based bloom filter adjustment mechanism to enhance the read performance of LSM-tree. Specifically, we introduce an asynchronous bloom filter memory estimation algorithm and propose a dynamic resource adjustment method through a defined multi-container data structure and bloom filter configuration unit configuration protocol. Experimental results demonstrate that our proposed system, i.e., RallyDB, reduces read latency by 25% compared to ElasticBF, which significantly improves database performance.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

LSTM-Based Query Performance Optimization in LSM-Trees

  • Yuxin Guo,
  • Shuhe Zhang,
  • Hongjuan Li,
  • Hui Kang

摘要

Recently, advancements in storage architectures and query optimization techniques have significantly improved the performance of database, driving the evolution of database systems. Moreover, the demand for database systems to quickly index and retrieve large volumes of data has increased substantially. The log-structured merge tree (LSM-tree), recognized for its excellent write performance, is particularly well-suited for NoSQL database due to its capability to handle large datasets and facilitate high-frequency writes and updates to indexes and data. However, the multi-level storage structure of LSM-tree necessitates searches in multiple sorted string tables during read operations, resulting in high disk I/O overhead and increased query times, which decreases the query efficiency of database systems. To address this challenge, we propose a machine learning-based bloom filter adjustment mechanism to enhance the read performance of LSM-tree. Specifically, we introduce an asynchronous bloom filter memory estimation algorithm and propose a dynamic resource adjustment method through a defined multi-container data structure and bloom filter configuration unit configuration protocol. Experimental results demonstrate that our proposed system, i.e., RallyDB, reduces read latency by 25% compared to ElasticBF, which significantly improves database performance.