Transformers have achieved significant success in the field of medical image segmentation, primarily due to their ability to capture long-range dependencies among pixels through the self-attention mechanism. However, Transformer models struggle with capturing fine-grained local details. In this work, we propose a Medical Image Sparse Segmentation Model (MedSparseSeg) incorporating a novel Spatial-Channel Aware Convolutional Sparse Coding (SC-CSC) decoder. MedSparseSeg utilizes a pre-trained multi-axis vision transformer (MaxViT) as the encoder to extract features, which are then processed by the SC-CSC decoder for image segmentation. In our decoder, channel and spatial attention mechanisms are combined to refine feature representations at both channel and spatial levels, resulting in a more holistic enhancement. Moreover, Multi-Layer Convolutional Iterative Shrinkage-Thresholding Algorithm modules (ML-ConvISTA) are used to enhance fine-grained feature extraction and local feature representation, emphasizing crucial details while suppressing noise. Experimental results demonstrate that our model achieves superior performance for medical image segmentation on the ACDC and Synapse datasets.