Skin lesion segmentation is a challenging task in computer-aided diagnosis, which is crucial for the early diagnosis of skin cancer. Convolutional Neural Networks (CNNs) have been successful in medical image segmentation tasks; however, their effective receptive fields in deep convolutional layers are limited to a local range and follow Gaussian distribution, thereby failing to obtain global information. Advanced Transformer shows great potential in modeling long-range dependencies and obtaining global representations. Therefore, we propose a multi-path segmentation model (MSNet) based on a combination of CNN and Transformer, which is dedicated to facilitating the task of skin lesion segmentation. Regarding different task requirements, we design MSNet-1 for the real-time tasks, and MSNet-2 for the tasks that require high accuracy. Moreover, we develop an efficient residual module (ERM) in MSNet, which can effectively integrate multi-level features and provide accurate feature representations. Pixel attention and coordinate attention are also introduced to enhance the perceptual ability of the network and improve the predicting accuracy of the segmentation results. Finally, we conduct extensive experiments on three public skin lesion datasets and one thyroid nodule dataset. The experimental results demonstrate that MSNet not only possesses the SOTA segmentation performance and excellent generalization ability, but also has lightweight and real-time characteristics, and it has broad application prospects in various scenarios.