Cardiovascular disease is one of the most difficult diseases to treat. Segmentation of cardiac images is an essential approach to improve the effectiveness and accuracy of cardiovascular disease treatment and intervention. Cardiac image segmentation, however, remains a challenging task, especially for effectively and accurately segmenting the multimodal images and simultaneously segmenting multiple heart substructures. In this paper, we propose a dual-branch U-shaped network based on wavelet transform, called DWU-Net. First, we convert the multimodal medical images into low-frequency and high-frequency images using wavelet transform to narrow the multimodal feature differences. The network consists of two branches of U-shaped sub-network, which accepting low-frequency and high-frequency images, respectively. Low-frequency and high-frequency features extracted by the corresponding branch are fused in deep levels and the final segmentation is achieved by both supervising on segmentation results from fused features and self-supervising among segmentation results from the two branches. The proposed model was verified on the Whole Heart Segmentation ++ (WHS++) dataset, showing greatly improved segmentation results compared with state-of-the-art methods.