Parkinson's disease (PD) is a prevalent neurodegenerative disorder characterized by progressive motor and non-motor dysfunctions. The inability of single modalities to capture the full spectrum of disease characteristics highlights the importance of multimodal approaches in PD prediction. Decision-level fusion is a critical technique for integrating multimodal heterogeneous data sources. However, traditional methods struggle to effectively assess the relative importance of each modality and feature, limiting their ability to evaluate overall disease progression and optimize diagnostic accuracy comprehensively. In this paper, we propose an end-to-end multimodal framework, Multimodal Gaussian Process Decision-level Fusion (MGPDF), to tackle challenges in multimodal data integration. The core innovation of MGPDF is its adaptive dynamic weight allocation mechanism, which uses Gaussian Process modeling to balance contributions from heterogeneous data modalities. Specifically, we introduce: 1) a pre-trained Gaussian Process Regressor (GPR) to predict modality-specific contributions, enabling flexible, data-driven weight distribution; and 2) a feature weighting strategy based on the mean and variance of probability matrices to capture overall trends while reducing noise. This approach effectively integrates multimodal data to enhance the model’s robustness and generalization. Comprehensive experiments on the YouTubePD dataset demonstrate that MGPDF achieves state-of-the-art performance in multimodal PD severity prediction.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

MGPDF: A Multi-modal Gaussian Process Decision-Level Fusion Model for Parkinson's Disease Prediction

  • Keyu Shen,
  • Xiaobo Zhang,
  • Yutao Liu,
  • Yan Yang,
  • Xiaole Zhao

摘要

Parkinson's disease (PD) is a prevalent neurodegenerative disorder characterized by progressive motor and non-motor dysfunctions. The inability of single modalities to capture the full spectrum of disease characteristics highlights the importance of multimodal approaches in PD prediction. Decision-level fusion is a critical technique for integrating multimodal heterogeneous data sources. However, traditional methods struggle to effectively assess the relative importance of each modality and feature, limiting their ability to evaluate overall disease progression and optimize diagnostic accuracy comprehensively. In this paper, we propose an end-to-end multimodal framework, Multimodal Gaussian Process Decision-level Fusion (MGPDF), to tackle challenges in multimodal data integration. The core innovation of MGPDF is its adaptive dynamic weight allocation mechanism, which uses Gaussian Process modeling to balance contributions from heterogeneous data modalities. Specifically, we introduce: 1) a pre-trained Gaussian Process Regressor (GPR) to predict modality-specific contributions, enabling flexible, data-driven weight distribution; and 2) a feature weighting strategy based on the mean and variance of probability matrices to capture overall trends while reducing noise. This approach effectively integrates multimodal data to enhance the model’s robustness and generalization. Comprehensive experiments on the YouTubePD dataset demonstrate that MGPDF achieves state-of-the-art performance in multimodal PD severity prediction.