Coastal vegetation is of great importance in attenuating coastal flooding and enhancing coastal disaster prevention. While numerous studies have examined wave attenuation by mangroves simplified using rigid cylinder models, limited research has focused on the interactions between extreme waves and mangroves with complex root systems, such as those of Rhizophora species. Prototype-scale experimental and CFD numerical models were conducted to investigate the protective performance of the Rhizophora mangrove forest under tsunami-like wave conditions. The influence of wave and mangrove characteristics on wave attenuation coefficients was systematically analyzed. Results indicate that higher incident wave heights and water depths generally reduce wave attenuation coefficients, while high-density mangrove forests (HDM) significantly enhance wave attenuation coefficients compared to low-density configurations (LDM). To predict the wave attenuation coefficient, five dimensionless parameters, derived from mangrove and wave characteristics, were evaluated using Artificial Neural Networks (ANN). The ANN model shows excellent predictive performance (R2 = 0.98) for wave attenuation coefficients, identifying mangrove density and cross-shore width as the most dominant factors, contributing 47% and 30% to wave attenuation, respectively. These findings emphasize the critical role of the mangrove forest in coastal flood mitigation and provide valuable insights for coastal planning and disaster risk reduction strategies in vulnerable regions.