Exploring Cross-Modality Fusion of Neuroimaging and Handwriting Biomarkers in Parkinson’s Disease

This paper proposes a deep learning-based cross-modality fusion framework that integrates neuroimaging data (MRI and DaTSCAN) with handwriting-derived image data to capture distinctive patterns of Parkinson disease. While neuroimaging modalities provide valuable insights into structural and functional brain abnormalities, handwriting-based assessments capture subtle motor dysfunctions that are also highly relevant to disease characterization. By combining structural brain features with quantitative handwriting characteristics, the proposed approach demonstrates that complementary information across modalities can be effectively leveraged. Following preprocessing and enhancement procedures applied to the images, important features were extracted using a ResNet50 model fine-tuned with the multimodal dataset. The obtained feature set was then used to train a stacking classifier. This ensemble-based strategy allows the model to leverage complementary strengths of different classifiers, providing a more robust decision mechanism. Experimental results demonstrate that this cross-modality integration is feasible and yields promising results, highlighting the potential of generating multimodal datasets for Parkinson research. Such datasets could enable the design of task-specific models in future studies, ultimately contributing to more comprehensive and reliable assessment.