AI Based Self-Managed Query Optimization Framework for Tightly Coupled Multistore Big Data Systems
摘要
Increasingly, modern data-intensive applications depend on heterogeneous data storage technologies to meet a range of diverse requirements associated with scalability, consistency, workload characteristics and latency constraints. When deployed in isolation, traditional database systems tend to be insufficient to support these different requirements and create bottlenecks in performance and inefficient use of resources. To handle these issues, this paper proposes AQOM (Adaptive Query Optimization with Q-learning for Multi-store), a reinforcement learning-based query mediator that dynamically routes queries to optimal datastores. Novel contributions include: (1) adaptive Q-learning with state-space features and dynamic reward weighting based on real-time system metrics; (2) query pattern learning using semantic hashing and Welford’s algorithm; (3) confidence-based selection with automatic exploration for unseen patterns; and (4) a query conversion layer. Experiments show 40–97% execution time improvements over static policies.