Multimodal Parkinson’s Disease Detection Using Entropygated Fusion and Explainable Deep Learning

Abstract Early detection of Parkinson’s disease (PD) remains challenging due to its gradual onset, heterogeneous symptoms and reliance on subjective clinical assessments. In this study, a robust and interpretable multimodal framework is proposed that integrates three complementary biomarkers: digitized spiral drawings, speech-derived time–frequency representations and structural magnetic resonance imaging (MRI) slices. Each modality is processed using a dedicated encoder, where convolutional neural networks are applied to spiral and speech inputs and a Vision Transformer is employed for MRI data. The outputs are combined using an entropy-guided fusion mechanism that adaptively assigns weights based on prediction confidence. To improve clinical interpretability, Gradient-weighted Class Activation Mapping (Grad-CAM) and a novel Localization Support Index (LSI) are incorporated to quantify the alignment between model attention and clinically relevant regions. The framework is designed to handle missing or degraded modalities, ensuring robustness in practical screening scenarios. Experimental evaluation on a subject-independent, classbalanced dataset demonstrates that the proposed multimodal approach outperforms single-modality models, achieving an accuracy of 98.42%, sensitivity of 98.17% and specificity of 98.67%. Additional ablation and statistical analyses validate the effectiveness of entropy-based fusion and diversity regularization. These results highlight the potential of interpretable multimodal learning for improving early PD detection while maintaining transparency and real-world applicability. The ViT-B/16 MRI encoder is initialized using ImageNet-pretrained weights and then finetuned on the MRI training subset. Keywords: Grad-CAM, MRI, Multimodal Learning, Parkinson’s Disease, Speech MFCC, Spiral Drawing.

Author(s): Jaya Singh*, Ranjana Rajnish, Deepak Kumar Singh
Volume: 7 Issue: 3 Pages: 629-642
DOI: https://doi.org/10.47857/irjms.2026.v07i03.011804