Deep reinforcement learning (DRL) has demonstrated significant potential for solving complex decision-making problems, including inventory control in large-scale supply chains. While DRL’s ability to handle vast state and action spaces and enable near real-time decision-making is compelling, its practical application in this domain so far has yielded mixed results. We reflect on the progress and persistent challenges of applying DRL to real-world inventory control. We explore the gap between initial enthusiasm and the realities of implementation, highlighting key obstacles that must be addressed to unlock DRL’s full potential. Despite these challenges, we argue that DRL remains a promising avenue to transform supply chains.