Rolling contact fatigue (RCF) is a typical failure mode of rail steels due to repeated cyclic contact loading. The construction of an RCF model based on damage mechanics can incorporate failure mechanisms related to microstructure. In this work, a combined plasticity model for ferrite with crystal plasticity and cementite with isotropic plasticity is presented to describe the plastic heterogeneity of the pearlite. While the mechanical damage behavior of grain boundaries is defined using a cohesive zone model that incorporates fatigue degradation. Then, the RCF lifetime and fatigue damage evolution of pearlitic rail steel are investigated by the designed RCF model. Compared with the previous linear elasticity and isotropic plasticity models, the fatigue lifetime of the combined plasticity model agrees better with the experimental value. The superiority of the combined plasticity model is that it can reflect the localized stress concentration induced by anisotropic deformation and therefore can characterize the uncertainty of fatigue damage. In addition, the surface spalling caused by the propagation of surface and subsurface cracks is also observed in the combined plasticity model. These results above demonstrate the applicability of the proposed model in the fatigue damage modeling and simulation of pearlitic rail steel.