A Multi-action Reinforcement Learning Framework via Pointer Graph Neural Network for Flexible Job-Shop Scheduling Problems with Resource Transfer

Intelligent manufacturing has become the focus in the era of Industry 4.0, and the main objective of this study is to address the Flexible Job-shop Scheduling Problem with Resource Transfer (FJSP-RT). To this end, the paper proposes a reinforcement learning framework with multiple decision-making actions (multi-action) based on multi-Pointer Graph Neural Network (multi-PGN) to meet the challenges of FJSP-RT. First, in order to efficiently express the environmental state of FJSP-RT, a heterogeneous disjunction graph is constructed, and a Heterogeneous Graph-based Multi-Model Network (HGMN) is used as the embedding representation method of the graph. Second, based on the multi-action decision-making process of FJSP-RT, multi-PGN is used to obtain and analyze effective information in heterogeneous graph to make high-quality scheduling decisions. In addition, the proposed multi-PGN is implemented using a new Proximal Policy Optimization that extends a single action to multiple actions, called multi-PPO. Through extensive experiments on instances, the research results show that this method has superior performance compared to traditional hand-designed scheduling rules. Furthermore, the method also shows strong potential in solving large-scale problems that arise in training.