The cache is a critical component that directly impacts the computational system’s performance and Quality of Service (QoS), with its effectiveness determined by the caching policy in use. An ideal caching policy must adapt to diverse workloads while approximating the optimal strategy. Recent advancements in caching strategies have focused on two paradigms: user access pattern prediction based on supervised learning and policy search methods based on Reinforcement Learning (RL). Supervised learning approaches often suffer from predictive inaccuracies. RL methods face challenges with large state spaces and insufficient exploration. Both paradigms lead to suboptimal performance and unbearable complexity in the decision making process. In this paper, we propose CARIS, a novel caching policy that integrates cache affinity estimation with a deep reinforcement learning agent. CARIS estimates the cache affinity of each user access, and focuses the agent’s exploration on accesses with higher predicted cache affinity, effectively reducing unnecessary exploration and addressing the limitations of both paradigms. Additionally, to reduce complexity in decision making, CARIS introduces an “empty action” mechanism during cache hits to reduce noise from irrelevant actions. Through comprehensive experiments with multiple datasets, CARIS achieves an improvement more than 294% in the average cache hit rates compared to the SOTA learning-based caching policies under the tested scenarios. Our results highlight CARIS’s potential to advance both theoretical understanding and practical efficiency in modern caching systems.