Early deepfakes primarily focused on visual face swapping, but the advancement of multimodal deepfake technology now allows for realistic face and audio replacements. Although some researchers have made advances in using multimodal learning for deepfake detection, they still encounter two major challenges: heterogeneity and complementary data fusion. We propose a novel approach called DDformer, and introduce two fusion methods: Multimodal Fusion Transformer (MFT) and Shared Weight Attention Fusion (SWAF). MFT utilizes the powerful global modeling capability of the transformer, which enhances the fusion of multimodal features. However, SWAF incorporates channel attention with shared weights to further complement and enhance multimodal features. Finally, we design a novel classifier specifically tailored for detecting different types of deepfakes. This classifier effectively utilizes the fused multimodal features to accurately classify and identify various types of deepfake videos. DDformer achieved a multi-class classification accuracy of 97.59% on the challenging FakeAVCeleb dataset and demonstrated its generalization ability through generalization experiments. DDformer provides a promising solution for addressing the challenges of heterogeneity and complementary data fusion in multimodal deepfake detection.