Recent advances in deep learning methodologies, particularly convolutional neural networks (CNNs) and transformer-based architectures, have driven remarkable advancements in the development of medical image segmentation systems. However, there are still some limitations in terms of segmentation accuracy and computational complexity. These issues can lead to reduced segmentation performance, especially in handling fine details and multi-scale information fusion. To overcome these challenges, we introduce DAMF-UNet, an innovative U-shaped architecture with dynamic attention mechanisms for precise medical image segmentation. Specifically, DAMF-UNet employs a Dual-attention Transformer module to establish a multi-scale encoder-decoder framework. Meanwhile, the network proposes a Feature Fusion Residual Module as a novel skip connection strategy. In addition, to enhance the global dimensional interaction of multi-scale features, DAMF-UNet also designed a Multi-scale Normalization Channel Attention module. We conducted extensive experimental evaluations on various datasets, including multi-organ CT and skin lesion images, and our method achieved state-of-the-art performance. Specifically, on the Synapse dataset, we achieve a Dice Similarity Coefficient (DSC) of 82.92% and a Hausdorff Distance at 95% (HD95) of 17.20. On the ISIC 2018 dataset, we achieve a DSC of 91.01%.