Accurate prediction of the shear strength ( \({\tau }_{p}\) ) of rock joints is essential for ensuring the stability and safety of geotechnical structures. This study introduces a novel framework for integrating the CatBoost gradient boosting decision tree algorithm with six cutting-edge metaheuristic optimization techniques, offering enhanced accuracy and reliability in shear strength prediction. The research employs advanced evaluation tools, including error metrics, Taylor diagrams, relative deviation distribution diagram, relative absolute error-cumulative frequency, and uncertainty analyses, to validate model robustness under diverse geological conditions. Among the optimized models, the CatBoost-Grey Wolf Optimizer (CatB-GWO) emerged as the most accurate, while the CatBoost-Whale Optimization Algorithm (CatB-WOA) demonstrated superior consistency and minimal bias. Sensitivity analysis identified normal stress ( \({\sigma }_{n}\) ) as the most influential parameter affecting shear strength. Unlike traditional approaches, this study combines computational intelligence and geomechanical insights to advance predictive modeling in rock mechanics, establishing a novel methodology for handling complex geotechnical challenges. These findings highlight the transformative potential of hybrid machine learning models in enhancing shear strength prediction for rock joints.
