<p>The U-shaped network architecture has become a foundational model in medical image segmentation. However, when dealing with the fusion of features at multi-resolution and introducing mechanisms to improve performance, it often generates a large amount of interfering information, which not only increases the model’s computational cost but also compromises its segmentation performance. To address this, we propose FFDA-Net, a novel Fourier fusion and dual-path attention enhancement network that adopts a U-shaped architecture design. FFDA-Net introduces a Fourier feature fusion module, which selectively retains relevant features in the skip connections while suppressing irrelevant ones, thereby alleviating the issue of information interference caused by semantic differences when fusing features at different resolutions. To further optimize feature representation, FFDA-Net designs an efficient dual-path attention feature enhancement module, which enhances the model’s ability to capture local spatial details. Additionally, to further reduce interfering information, FFDA-Net adopts a novel hybrid loss function and employs a fully supervised training strategy. Experimental results demonstrate that FFDA-Net achieves excellent segmentation performance on public medical image segmentation datasets across four different medical imaging modalities. On the Synapse dataset, FFDA-Net achieves advanced performance with the highest average Dice score of 84.51% and the highest mean Intersection over Union (mIoU) of 75.72%, while also offering significant advantages in computational cost.</p>