Multi-objective optimization enabling CFRP energy-efficient milling based on deep reinforcement learning

The expanding application of Carbon Fiber Reinforced Polymer (CFRP) in industries is drawing increasing attention to energy efficiency improvement and cost reducing during the secondary processing, particularly in milling. Machining parameter optimization is a practical and economical way to achieve this goal. However, the unclear milling mechanism and dynamic machining conditions of CFRP make it challenging. To fill this gap, this paper proposes a DRL-based approach that integrates physics-guided Transformer networks with Twin Delayed Deep Deterministic Policy Gradient (PGTTD3) to optimize CFRP milling parameters with multi-objectives. Firstly, a PG-Transformer-based CFRP milling energy consumption model is proposed, which modifies the existing De-stationary Attention module by integrating external physical variables to enhance modeling accuracy and efficiency. Secondly, a multi-objective optimization model considering energy consumption, milling time and machining cost for CFRP milling is formulated and mapped to a Markov Decision Process, and a reward function is designed. Thirdly, a PGTTD3 approach is proposed for dynamic parameter decision-making, incorporating a time difference strategy to enhance agent training stability and online adjustment reliability. The experimental results show that the proposed method reduces energy consumption, milling time and machining cost by 10.98%, 3.012%, and 14.56% in CFRP milling respectively, compared to the actual averages. The proposed algorithm exhibits excellent performance metrics when compared to state-of-the-art optimization algorithms, with an average improvement in optimization efficiency of over 20% and a maximum enhancement of 88.66%.