Optimal Scheduling of an Integrated Electricity–Heat–Hydrogen Energy System Based on an Improved TD3 Algorithm

To enhance the synergistic utilization of hydrogen, electricity, and heat within integrated energy systems, improve the flexibility of energy allocation, and reduce carbon emissions, this paper proposes an operational optimization strategy for the Electricity-Heat-Hydrogen Coupled Integrated Energy System (EHHC-IES). The mathematical models of the various devices in the EHHC-IES are established, and the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm is improved by replacing the conventional Ornstein–Uhlenbeck (OU) noise exploration with State-Dependent Exploration (SDE). The uncertainty-based optimal scheduling problem of the EHHC-IES is formulated as a Markov Decision Process (MDP), and the improved TD3 algorithm is employed to jointly encode the optimization objectives and system constraints into the reward function. Dynamic scheduling decisions are then carried out in continuous state and action spaces to generate rational energy allocation strategies. By training the agent using historical operational data and comparing the results with those obtained by the standard TD3 algorithm, the study demonstrates that the scheduling scheme based on the improved TD3 achieves better economic performance, with dispatch costs closer to the day-ahead optimization results solved by CPLEX. Moreover, it exhibits greater adaptability and effectiveness in dynamic optimization scheduling, contributing to enhanced flexibility, low-carbon operation, and cost reduction of the integrated energy system.