Just as U-Net, most of the classical medical image segmentation models are U-shaped network structures, and most of them use direct skip connection structures. However, it is still challenging to obtain the global information of features using this simple skip connection structure. Because of the semantic gaps between the features from encoders and decoders, direct skip connections are often not very efficient. Mamba, as a state-space sequence model (SSM) with linear complexity for modeling long-rang dependency, emerged recently. In this paper, we use Mamba as the main structure to reconstruct the skip connection structure, and propose the Residual Mamba Block (RMB) as a new skip connection module, which is the first module integrating Mamba into skip connection. Specifically, we use RMB to model the remote dependencies of the features from the encoder, and then fuse them with the features from the decoder, so as to obtain features that are more suitable for the features from the decoder, reduce the semantic gaps between features, and improve the performance of the model. Mamba is used as the main structure of the new skip connection module, mainly because of its linear complexity and ability to model remote dependencies. Therefore, with only a slight increase in the amount of computation, the segmentation performances of the models can be improved when using it in original models. The experimental results show that the proposed RMB skip connection structure can improve the segmentation performance of U-Net and its variants on multiple medical image datasets.