<p>Multimodal learning has recently gained considerable attention in fault diagnosis for its ability to integrate multisource heterogeneous data and extract comprehensive fault features. However, existing multimodal fault diagnosis studies have the following limitations: (1) existing multimodal methods lack flexibility in handling arbitrary modality combinations under missing-modality conditions, thereby limiting their applicability in practical industrial environments; (2) disparities in data distributions and feature scales across modalities often lead to imbalanced convergence during training, hindering the realization of its theoretical potential. To address these issues, a flexible multimodal framework with inter-modal learning discrepancy mitigation is proposed for motor fault diagnosis under modal missing scenarios. Firstly, a flexible sparse mixture of experts framework is designed to adaptively integrate arbitrary modality combinations while maintaining robustness against missing data. Then, a discrepancy-aware alignment scheme grounded in contrastive learning is introduced to harmonize fault-category representations across modalities, bridging heterogeneous features and mitigating inter-modal learning gaps. Finally, regularization-guided synergistic loss optimization is developed to dynamically integrate unsupervised contrastive learning with multimodal representation learning, which enables more effective cross-modal feature fusion. The effectiveness of the proposed method is validated through simulated motor fault experiments, and its superiority is demonstrated by comparisons with advanced approaches for multimodal missing and imbalance scenarios.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

A flexible multimodal motor fault diagnosis framework with inter-modal learning discrepancy mitigation under modal missing scenarios

  • Yuhan Xie,
  • Yuewei Xi,
  • Biao Wang,
  • Xiaoqing Cheng,
  • Liang Guo,
  • Yong Qin

摘要

Multimodal learning has recently gained considerable attention in fault diagnosis for its ability to integrate multisource heterogeneous data and extract comprehensive fault features. However, existing multimodal fault diagnosis studies have the following limitations: (1) existing multimodal methods lack flexibility in handling arbitrary modality combinations under missing-modality conditions, thereby limiting their applicability in practical industrial environments; (2) disparities in data distributions and feature scales across modalities often lead to imbalanced convergence during training, hindering the realization of its theoretical potential. To address these issues, a flexible multimodal framework with inter-modal learning discrepancy mitigation is proposed for motor fault diagnosis under modal missing scenarios. Firstly, a flexible sparse mixture of experts framework is designed to adaptively integrate arbitrary modality combinations while maintaining robustness against missing data. Then, a discrepancy-aware alignment scheme grounded in contrastive learning is introduced to harmonize fault-category representations across modalities, bridging heterogeneous features and mitigating inter-modal learning gaps. Finally, regularization-guided synergistic loss optimization is developed to dynamically integrate unsupervised contrastive learning with multimodal representation learning, which enables more effective cross-modal feature fusion. The effectiveness of the proposed method is validated through simulated motor fault experiments, and its superiority is demonstrated by comparisons with advanced approaches for multimodal missing and imbalance scenarios.