Reinforcement Learning for Time-Series Query Optimization

Time-series databases (TSDBs) are essential for managing large-scale time-series data in fields like finance, IoT, and agriculture. However, traditional query optimization methods, such as dynamic programming, struggle with high computational complexity and inaccurate cost estimates. This paper proposes a novel query optimization module for TSDBs using reinforcement learning (RL), specifically Deep Q-Networks (DQN) and Double Deep Q-Networks (DDQN). These algorithms dynamically learn optimal join orders based on query workloads and connection costs. Experiments show that RL-based methods achieve better optimization performance and stability compared to traditional heuristics, especially under complex cost models. This work highlights the potential of RL in improving query optimization for TSDBs.