<p>Rock slopes are primarily susceptible to deformation and failure due to the development of fissures and the formation of cavities at their base. By simplifying the unstable rock mass into a rock beam and applying the cantilever beam model, the mechanical governing equation of the rock beam resting on a Pasternak foundation was constructed. Using the finite difference method, we proposed a novel approach to calculate the internal forces of unstable rock masses and thoroughly examined the mechanical properties of rock beams influenced by various factors, including cracks, rock cavities, rock mass geometry, loads, and rock beam stiffness. Finally, by integrating the classical cantilever beam theory, we established a criterion for assessing the toppling failure of unstable rock masses. The study revealed that the emergence of dissolved cavities resulted in stress concentration at the cavity boundaries, and significant increases were observed in the maximum deformation, bending moment, and shear force. Furthermore, an increase in the dip angle, natural slope angle, and slope height augmented the sliding force on the slope. In contrast, a negative correlation existed between the thickness of the rock stratum and the sliding force. A decrease in the applied load and an increase in the stiffness of the rock stratum contributed to the higher self-stability of the rock beam. However, a reduction in the thickness of the rock stratum and an increase in the length of dissolution cavities resulted in a decrease in the stability coefficient of the unstable rock mass. These findings provide a comprehensive understanding of the factors influencing the stability of unstable rock slopes and their associated mechanical behaviors.</p>