A simulation-based digital twin for data-driven maintenance scheduling of risk-prone production lines via actor critics

Industry 4.0 mandates a shift from traditional total productive maintenance (TPM) methods, which rely on periodic data gathering and subsequent offline modeling for maintenance scheduling, towards more data-driven and online decision-making approaches. Further, Industry 4.0 emphasizes reducing intervention from high-skilled managers and leveraging real-time data for decision-making, which is characteristic of models rooted in either renewal-theoretic or Markov chains for traditional TPM methods. Digital Twins (DTs), which are virtual representations of physical systems, play a crucial role in this paradigm by enabling online decision-making of maintenance scheduling directly at the workstation level. In this paper, a simulation-based DT (S-DT) is designed that bypasses offline modeling for individual workstations as well as production lines. It uses data drawn from real time and works to prioritize and customize key maintenance tasks, based on evolving urgency of maintenance tasks and the risk of disruptive failures and missed production targets associated to irregular maintenance. Further, it determines a near-optimal maintenance schedule using a novel actor-critic algorithm from the domain of reinforcement learning for average-reward semi-Markov decision processes. The framework is validated numerically in an S-DT, demonstrating its ability to rapidly determine maintenance schedules for individual workstations and risk-prone production lines, such as in the ball-bearings industry. The results highlight the potential of this approach for online implementation, which aligns with Industry 4.0 initiatives.