Deep Reinforcement Learning Based on Graph Neural Network for Flexible Job Shop Scheduling Problem with Lot Streaming

The flexible job shop scheduling problem with lot streaming (FJSPLS) has gained considerable attention due to its potential to significantly reduce manufacturing completion time. FJSPLS couples three important sub-problems: operation sequencing, machine selection, and lot splitting. Deep reinforcement learning(DRL) methods have not been applied to this research area. This paper presents an innovative method using DRL with graph neural network (GNN) to solve FJSPLS with equal and consistent sub-lots. First, to facilitate integrated decision-making for the sub-problems, two distinct agents with independent parameters are developed. One agent is designated to handle the integrated decisions concerning operations arrangement and machine allocation, while the other focuses on overseeing decisions regarding lot-splitting for job orders. Second, to enhance the capabilities of each agent in extracting features from environmental observations, an enhanced disjunctive graph (EDG) and heterogeneous graph feature extractor(HGFE) have been constructed. The EDG comprises three types of nodes, encompassing job nodes, operation nodes, and machine nodes. Then, the proposed network architecture uses the proximal policy optimization (PPO) algorithm for better performance realization. Finally, the experimental findings confirm the effectiveness of the proposed scheduling approach in reliably solving the FJSPLS.