PPO-based deepcluster agent: efficient determination of global minima structures in Ag–Cu nanoalloys by deep reinforcement learning
摘要
The computational discovery of global minimum structures in bimetallic nanoclusters is a fundamental challenge in materials science, hindered by the exponentially complex and rugged nature of their potential energy surfaces. Traditional global optimization methods, such as genetic algorithms, often struggle with premature convergence and computational inefficiency. Here, we utilized a deep reinforcement learning (DRL) framework that autonomously and efficiently navigates the PES of Ag–Cu 13-atom nanoclusters across the entire compositional range (Ag13−mCum, m = 0–13). The agent, trained using a Trust Region Policy Optimization algorithm, learns an optimal policy for structural exploration by strategically displacing atoms to discover lower-energy minima while actively avoiding unphysical states. The framework demonstrates a clear learning progression, evolving from random exploration to a highly efficient search strategy, consistently identifying five distinct low-energy minima in as few as 5–7 steps. Crucially, the DRL-discovered global minimum structures are consistently more stable—by 19–98 meV/atom—than those located by a benchmark genetic algorithm, as validated by both effective medium theory and density functional theory calculations. Furthermore, a Proximal Policy Optimization (PPO)-based Deepcluster agent reproduces the same global minima with up to 11-fold fewer training episodes (e.g., 550 vs. 6000 episodes for Cu13), demonstrating a substantial improvement in computational efficiency. This work establishes DRL as a superior, autonomous paradigm for the inverse design of nanoalloys, offering a robust and transferable strategy for accelerating the discovery of functional nanomaterials with tailored properties.
Graphical Abstract