Scholarlink

<p>The problem of efficient slotting in the warehouse under multiple objectives and constraints is still an open problem, especially in high-velocity environments, taking into account the weight of items, their frequency of retrieval, and the duration of storage. The existing rule-based and conventional reinforcement learning techniques and methodologies have shown limited potential in efficiently solving this problem, considering the spatial and semantic dependencies and the decision structure under the influence of the imposed constraints. In this paper, a novel reinforcement learning framework, namely DTQNetwork, is introduced, which incorporates a multi-head transformer encoder with a Double Deep Q-Network. The framework is targeted at developing an intelligent system for pallet placement in the warehouse. The introduced framework incorporates the existing constraints, including weight stability, urgency, and frequency, using a carefully designed reward and penalty system. The framework is implemented and tested using a custom OpenAI Gym environment, simulating a dual-section <InlineEquation ID="IEq1"> <EquationSource Format="TEX">\(14 \times 14\)</EquationSource> </InlineEquation> warehouse grid with Automated Guided Vehicle operations. The results show significant improvements over existing methodologies, including a 60<InlineEquation ID="IEq2"> <EquationSource Format="TEX">\(\%\)</EquationSource> </InlineEquation> improvement in slot utilization and 85<InlineEquation ID="IEq3"> <EquationSource Format="TEX">\(\%\)</EquationSource> </InlineEquation> reduction in time. The framework is expected to provide a solution for developing intelligent storage systems, considering the existing and emerging challenges and complexities. First, we integrate a multi-head Transformer encoder into a Double DQN pipeline for smart warehouse slotting. Second, we design a custom Gym-based simulation framework that enforces physical and operational constraints during both training and inference. Third, through detailed benchmarking and ablation studies, we demonstrate the applicability of DTQNetwork to real-world warehouse optimization.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Reinforcement learning for smart warehouse systems using transformer-augmented double deep Q-networks

  • 期刊论文

  • 发表日期：

    2026年6月3日

查看全文
  • Hitesh Reddy Dereddy,
  • Sankeerth Latheesh,
  • Kanav Jeet Singh,
  • Narayan C. Debnath,
  • Garima Aggarwal,
  • Ngoc Huan Le,
  • Van Luan Tran

摘要

The problem of efficient slotting in the warehouse under multiple objectives and constraints is still an open problem, especially in high-velocity environments, taking into account the weight of items, their frequency of retrieval, and the duration of storage. The existing rule-based and conventional reinforcement learning techniques and methodologies have shown limited potential in efficiently solving this problem, considering the spatial and semantic dependencies and the decision structure under the influence of the imposed constraints. In this paper, a novel reinforcement learning framework, namely DTQNetwork, is introduced, which incorporates a multi-head transformer encoder with a Double Deep Q-Network. The framework is targeted at developing an intelligent system for pallet placement in the warehouse. The introduced framework incorporates the existing constraints, including weight stability, urgency, and frequency, using a carefully designed reward and penalty system. The framework is implemented and tested using a custom OpenAI Gym environment, simulating a dual-section \(14 \times 14\) warehouse grid with Automated Guided Vehicle operations. The results show significant improvements over existing methodologies, including a 60 \(\%\) improvement in slot utilization and 85 \(\%\) reduction in time. The framework is expected to provide a solution for developing intelligent storage systems, considering the existing and emerging challenges and complexities. First, we integrate a multi-head Transformer encoder into a Double DQN pipeline for smart warehouse slotting. Second, we design a custom Gym-based simulation framework that enforces physical and operational constraints during both training and inference. Third, through detailed benchmarking and ablation studies, we demonstrate the applicability of DTQNetwork to real-world warehouse optimization.