Scholarlink

This paper proposes an optimization approach for supervised control of stochastic timed discrete event systems (STDES) that integrates reinforcement learning (RL) with supervisory control theory (SCT). Unlike crisp discrete event systems (CDES) with disregard for time, STDES analyzes more complex dynamic systems, which consider general distributions for sojourn time between states. So STDES is closer to complex physical systems and its optimal control problem is more challenging. In this paper, the language of the uncontrolled system and its deterministic specification are firstly generated by automata according to the logic features of STDES system; Then, a supervisor is derived using SCT to ensure safety control by disabling specific controllable event sequences; Next, the controlled automata structure from SCT is converted into a Semi-Markov Decision Process(SMDP) framework by adding the temporal and probabilistic variable of the uncontrollable events. Finally, a modified SCT-Timed RL adaptive control algorithm is developed to optimize system performance by designing a new Bellman equation and reward function. The results show that the proposed controller enhances system safety and flexibility, with an 8.9 \(\%\) performance improvement over non-intelligent method.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Optimized Supervised Control of Stochastic Timed Discrete Event Systems Using Supervisory Control Theory and Reinforcement Learning

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Yingjun Liu,
  • Fuchun Liu,
  • Hongzhen Zhu

摘要

This paper proposes an optimization approach for supervised control of stochastic timed discrete event systems (STDES) that integrates reinforcement learning (RL) with supervisory control theory (SCT). Unlike crisp discrete event systems (CDES) with disregard for time, STDES analyzes more complex dynamic systems, which consider general distributions for sojourn time between states. So STDES is closer to complex physical systems and its optimal control problem is more challenging. In this paper, the language of the uncontrolled system and its deterministic specification are firstly generated by automata according to the logic features of STDES system; Then, a supervisor is derived using SCT to ensure safety control by disabling specific controllable event sequences; Next, the controlled automata structure from SCT is converted into a Semi-Markov Decision Process(SMDP) framework by adding the temporal and probabilistic variable of the uncontrollable events. Finally, a modified SCT-Timed RL adaptive control algorithm is developed to optimize system performance by designing a new Bellman equation and reward function. The results show that the proposed controller enhances system safety and flexibility, with an 8.9 \(\%\) performance improvement over non-intelligent method.