Scholarlink

<p>Modern underwater warfare requires sophisticated torpedo countermeasure strategies to ensure mission success in complex threat environments. Previous hierarchical reinforcement learning (HRL) approaches show limited scenario adaptability, training efficiency, and structural flexibility. In this paper, we propose Multiple Specialized Low-Level Agent Hierarchical Reinforcement Learning (MSLA-HRL) for submarine torpedo countermeasures, featuring a 1:N architecture where a strategic high-level agent coordinates N specialized low-level agents. Each low-level agent independently develops expertise for specific threat scenarios, while the high-level agent performs strategic selection and decoy deployment planning. In order to train this hierarchical structure, we integrate Active Curriculum Learning (ACL) with three modules: tactical guideline-based Scoring, Adaptive Pacing, and continuous Score Update. Our ACL dynamically adapts to evolving agent capabilities, enabling stable convergence in sparse reward environments with non-stationary dynamics inherent in hierarchical learning. Experimental validation across 630 torpedo threat scenarios demonstrates that the complete MSLA-HRL framework with ACL achieves a 97% mission success rate, representing a 17.8%p improvement over our previous 1:1 hierarchical model. The results indicate improvements in training stability and operational flexibility. This work suggests that adaptive hierarchical systems with curriculum-based training can mitigate key limitations of reinforcement learning in underwater warfare scenarios.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Hierarchical reinforcement learning with low-level agents and active curriculum for torpedo countermeasures

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年8月4日

查看全文
  • BoSeon Kang,
  • YooJung Yoon,
  • JiWoong Choi,
  • WonHyuk Yun

摘要

Modern underwater warfare requires sophisticated torpedo countermeasure strategies to ensure mission success in complex threat environments. Previous hierarchical reinforcement learning (HRL) approaches show limited scenario adaptability, training efficiency, and structural flexibility. In this paper, we propose Multiple Specialized Low-Level Agent Hierarchical Reinforcement Learning (MSLA-HRL) for submarine torpedo countermeasures, featuring a 1:N architecture where a strategic high-level agent coordinates N specialized low-level agents. Each low-level agent independently develops expertise for specific threat scenarios, while the high-level agent performs strategic selection and decoy deployment planning. In order to train this hierarchical structure, we integrate Active Curriculum Learning (ACL) with three modules: tactical guideline-based Scoring, Adaptive Pacing, and continuous Score Update. Our ACL dynamically adapts to evolving agent capabilities, enabling stable convergence in sparse reward environments with non-stationary dynamics inherent in hierarchical learning. Experimental validation across 630 torpedo threat scenarios demonstrates that the complete MSLA-HRL framework with ACL achieves a 97% mission success rate, representing a 17.8%p improvement over our previous 1:1 hierarchical model. The results indicate improvements in training stability and operational flexibility. This work suggests that adaptive hierarchical systems with curriculum-based training can mitigate key limitations of reinforcement learning in underwater warfare scenarios.