<p>Soft-hard interlayered rock mass represents one of the primary occurrences of layered rock formations. Numerous anti-dip slopes consisting of such strata in the reservoir areas of large-scale hydropower projects in Western China have undergone toppling failures. This study investigates the toppling deformation and failure of layered composite rock masses in anti-dip slopes by establishing two typical mechanical models for three-layer composite rock structures: Hard-Soft-Hard (HSH) and Soft-Hard-Soft (SHS). Through integrated theoretical derivations and numerical simulations, the role and effects of soft rock layers (i.e., the soft-layer effect) within these composite structures were systematically analyzed. Based on cantilever beam theory, analytical formulas for calculating the deflection and rotation angle of the composite rock mass under varying soft-to-hard rock thickness ratios were derived, with six configurations for the SHS structure (thickness ratios of 1:2, 1:1, 2:3, 1:2, 2:5, and 1:3) and six configurations for the HSH structure (thickness ratios of 2:1, 1:1, 3:2, 2:1, 5:2, and 3:1). These formulas were validated against numerical models. The results demonstrate that soft rock significantly influences the toppling deformation behavior. Specifically, the toppling deformation of both HSH and SHS composite rock masses exhibits an inverse correlation with the thickness variation of the middle rock layer. Numerical simulations show excellent agreement with theoretical predictions, with errors remaining below 4.44%. The soft-layer effect and its mechanical implications revealed in this study provide a theoretical foundation for analyzing toppling disaster mechanisms, assessing stability, and designing preventive measures for soft-hard interlayered rock masses.</p>