Towards online training for RL-based query optimizer

Join query optimization aims to find the best join order for tables in a query, which is critical for query processing performance. Recently, reinforcement learning models have been proposed to solve the challenges existing with query processing and join query optimization. However, changes in the data distribution can turn the trained reinforcement learning models into obsolete models, resulting in longer execution times. In this paper, we propose a new training strategy in order to extend the existing reinforcement learning models and improve their adaptation when the data distribution changes. The experiments show that the proposed strategy has a significant benefit in decreasing training time for the models given the changes in the data distribution.