The rising incidence of skin cancer necessitates advanced diagnostic and management strategies. Integrating modern computer vision techniques, particularly deep learning models, into clinical workflows can significantly improve early detection and diagnosis. This paper introduces the Spatial and Channel Attention with Dilation U-Net (SCA-DU-Net), an innovative end-to-end trainable network designed specifically for skin cancer segmentation. The proposed architecture improves upon the traditional U-Net by incorporating spatial attention, channel attention, and dilation mechanisms. This design efficiently captures both spatial and contextual information. The channel attention mechanism highlights key feature channels, while spatial attention focuses on critical areas around lesion boundaries. Dilation convolutions expand the receptive field while preserving spatial resolution. This enables the network to gather information across multiple scales. Evaluations on the ISIC 2018 dataset show that SCA-DU-Net outperforms state-of-the-art models, achieving 81.20% mean IoU, 90.20% Dice score, and 95.81% accuracy.