Dual-information driven deep multi-view clustering for multi-modal data
摘要
Deep multi-view clustering (DMVC) aims to utilize the consistency of multi-view data to learn a consensus representation using deep learning-based methods. However, existing methods overlook the presence of both content feature and topological structure information in the data. Also, the importance of these two information varies for multi-modal data. To address these issues, we propose Dual-Information Driven Deep Multi-View Clustering for multi-modal data (DID-DMVC). Firstly, to capture both content feature and topological structure information, we design a Dual-Information Extractor (DIE), which independently extracts two types of information. Secondly, we have designed a Tensor-Guided Low-Rank Fusion (TGLRF) strategy and developed a Dual-Level Adaptive Fusion Module (DLAFM). It adapts the fusion process by considering both the importance between views and the importance of the two different types of information for multi-modal data. Thirdly, we design an Auxiliary Contrastive Loss (ACL) to regulate discrepancies between content feature and topological structure information during the DLAFM process. Extensive experiments on various image, text and multimodal datasets demonstrate that our method consistently outperforms state-of-the-art baselines, confirming its effectiveness, robustness, and strong generalizability.