<p>Block theory is widely used for analyzing rock slope stability influenced by discontinuities. However, traditional methods often fail to account for the rotation of unstable blocks and the reconfiguration of unstable bodies following key block failures. To address these limitations, this paper proposes a novel method for quantitatively identifying boundaries and movement modes of potential unstable bodies in rock slopes. This method employs key blocks as basic units, systematically searching for movable block combinations from the slope surface inward, within a defined critical circular boundary to limit excessive combinations. Furthermore, unstable blocks are decomposed into spatial tetrahedral combinations, simplifying the mechanical analysis of their movement modes. The effectiveness of the method is demonstrated by its ability to identify potential unstable bodies with arbitrary shapes and various movement modes of unstable blocks, such as rotation and sliding. By considering the mass of each tetrahedron, this method also optimizes the initial movement direction of the entire unstable body. The applicability of the method was validated through a case study of an open-pit mine slope in Xinjiang, China, where four potential unstable bodies were identified. These findings assist in pinpointing critical reinforcement areas and assessing disaster risks. Overall, this method provides a valuable tool for mitigating slope-related hazards.</p>