Defending against multiple IIoT attackers using hybrid hypergame-reinforcement learning with ML-enhanced beliefs

Industrial Internet of Things (IIoT) systems face complex threats from multiple concurrent attackers who employ diverse strategies under incomplete information about defender capabilities. Conventional game-theoretic cyber-deception models rely on fixed strategy equilibria and static beliefs, limiting adaptability to evolving attacks. Meanwhile, pure reinforcement learning approaches lack strategic reasoning about adversarial objectives. This paper introduces a hybrid architecture combining hypergame theory (a game-theoretic framework where players may hold different perceptions of the game structure) and deep reinforcement learning for adaptive IIoT defense. Beliefs are updated using Random Forest classifiers trained on the CIC-IIoT 2025 dataset (500,000+ samples, 71 features, 8 attack categories), achieving 98.2% classification accuracy. The defender selects strategies via hypergame Nash equilibrium across four defense bundles and eight attacker strategies, with payoff matrices grounded in ML-derived detection rates. A Proximal Policy Optimization (PPO) meta-learner refines these equilibrium strategies through online interaction, balancing game-theoretic structure with adaptive learning. Experiments over 200 episodes show that our framework achieves 71.8% detection rate with 3.8% false positives against two simultaneous attackers, compared to static hypergame solutions (64.9% DR, 5.3% FPR) and non-learning baselines (35–56% DR, 6–9% FPR). To the best of our knowledge, this is one of the first approaches to integrate game theory, machine learning, and reinforcement learning for proactive defense under belief uncertainty.