<p>To address the challenge of balancing sample efficiency and control performance in reinforcement learning control of soft robots, this paper proposes a physics-informed reinforcement learning algorithm. The proposed method uses a deep Lagrangian-informed neural network for dynamics model learning and integrates it into a Dyna framework. At the model learning level, the Lagrangian structure of the soft robot is embedded into the neural network so that the learned dynamics model follows its underlying physical principles and serves as the environment model for subsequent learning. Physical priors derived from the Lagrangian formulation provide structural constraints during training, enabling the environment model to achieve higher accuracy with fewer samples and improving trajectory prediction performance. At the behavior learning level, policy learning adopts an actor–critic structure. Stronger policy optimization algorithms are further introduced to evaluate how policy optimization affects the overall framework performance. To validate the proposed method, a simulation environment is built in SoMoGym and ablation studies are conducted on both the model learning and behavior learning components. Results show that the proposed algorithm exhibits improved training stability and sample efficiency. For model learning, the deep Lagrangian-informed dynamics model achieves lower trajectory prediction errors than a purely data-driven deep neural network baseline, indicating that embedding physics information improves model accuracy and predictive performance. Compared with conventional model-based and model-free reinforcement learning baselines, the proposed algorithm attains lower control errors and better generalization in target domain randomization control tasks. These results demonstrate the effectiveness of the proposed algorithm for reinforcement learning control of soft robots.</p>