Cross-spectral stereo matching opens up a new route for disparity estimation in low-light scenes. However, the variations in light absorption and reflection properties for objects at different wavelengths create substantial domain gaps in cross-spectral image pairs. Accurately matching pixel pairs with significant domain gaps remains challenging. In this work, we propose a novel lightweight cross-spectral stereo matching network, dubbed CRFENet, exploiting reciprocal feature evolution. It enables the complementary evolution of the disparity prediction network and spectral translation network. To facilitate feature interactions between different domains to improve disparity results in regions with significant domain gaps, we design a domain mutual rail (DMR) module, which integrates monocular spectral and binocular cross-spectral features. Furthermore, we present a domain gap consistency constraint and corresponding confidence-weighted loss function, which aim to alleviate the matching challenge in regions with significant domain gaps through decoupling domain gap and disparity. Experimental results indicate that our CRFENet achieves state-of-the-art performance on the PittsStereo RGB-NIR and DrivingStereo datasets, especially for challenging regions with significant domain gaps. Besides, the model parameters are only 3.6M, which means high practicality.