The detection of material weaknesses in internal control (MWIC) of companies is essential since it provides early warning for various financial risks. Currently machine learning methods have been extensively adopted in this field. However, previous studies have not fully captured the relationships among different modalities, and neglected the consistency and complementary information. To address these challenges, we propose a Consistency and Complementary information-aware Multi-modal Deep Learning method (namely, CCMDL), which effectively captures the consistency and complementary relationships among different modalities. CCMDL comprises three main components, namely, the multi-modal feature extraction, the multi-modal feature enhancement and multi-modal feature fusion. Initially, multi-modal feature extraction module extracts valuable features and transforms them into deep representations. Subsequently, multi-modal feature enhancement module captures both the consistency and complementary information between these modalities, leveraging the joint effect of multi-modalities for MWIC detection. Finally, multi-modal feature fusion module introduces a novel pooling mechanism to detect MWIC. Through the experimental results on the real-world dataset, the proposed CCMDL outperforms the benchmark method in detecting MWIC. Our method enhances the effectiveness of detecting MWIC and provides robust decision-making support for regulators and investors.