Crystal structure prediction (CSP) is a central problem in materials science, aiming to determine the most stable atomic arrangement for a given chemical composition. In this study, we propose a no-data-driven CSP method using model-free reinforcement learning (RL). We design a novel reinforcement learning environment, CSP-Gym, in which the state is defined by the lattice parameters and atom positions of the crystal structure where actions involve tuning lattice parameters and moving atoms, with rewards based on the energy of the structure after local energy minimization, which guides the agents to explore low-energy configurations. Compared to traditional data-driven methods, this approach does not require pre-collected static datasets but dynamically generates data by interacting with the environment, demonstrating higher flexibility. CSP-Gym supports the integration of customized potential energy functions, and in this study, we implement the integration of LAMMPS, which we validated on various crystalline systems. Experimental results show that the method can efficiently search complex energy landscapes and discover stable structures, achieving energy errors of 7.43 × 10−5 eV/atom in CSP-Gym-Ar and 4.00 × 10−1 eV/atom in CSP-Gym-SiC when applying the optimal RL algorithm. We also conducted comparative experiments with cutting-edge data-driven models, demonstrating the effectiveness of RL algorithms and opening new paths for CSP task material science. The CSP-Gym environment has been open-sourced and is hosted at GitHub with documentation.