Autonomous Data Tiering Using Reinforcement Learning for 3-Tier Hierarchical Storage
摘要
In today’s data-driven world, organizations must efficiently store and manage massive amounts of information. As data generation keeps increasing, the demand for scalable and cost-effective storage solutions becomes critical. Data tiering appears to be a strategic solution to this problem by categorizing data based on access frequency and storing it across multiple storage tiers. This paper presents a new solution for autonomous data tiering using reinforcement learning (RL) within a 3-tier hierarchical storage system. By utilizing the adaptability and learning capabilities of RL, our approach dynamically optimizes data placement, ensuring high-performance access for frequently used data while lowering costs for infrequently accessed data. Our solution integrates an RL agent that is trained to recognize access patterns and make data placement decisions in real-time. This dynamic tiering strategy reduces latency while maintaining low costs. Through extensive experimentation and analysis, we show the effectiveness of our approach, highlighting its potential to transform the way data is handled in large-scale storage systems.