Purpose (i) To develop a lightweight custom model named Parkinson disease (PD) Net for the categorization of PD and normal brain image and compare its performance with VGG16 and VGG19 models in Magnetic Resonance imaging (MRI) images; (ii) to extract statistical textural and shape features and perform feature fusion with automated features and perform further classification. Materials and Methods In study, a dataset consisting of 30 MRI brain images (15 Parkinson’s positive (PP), and 15 Parkinson’s negative (NP)) in DICOM format was utilized. Additionally, 170 MRI brain images (85 PP, 85 NP) sourced from the International Neuroimaging Data share were also included in the analysis. The MRI brain image underwent segmentation through the skull stripping method, and subsequent statistical and brain features was extracted. The automated features derived from the PD Net models features were combined with extracted statistical features. The fused feature set was utilized for the classification into Parkinson’s disease and normal cases. Results The lightweight custom model named PD Net demonstrated superior performance compared to the VGG16 and VGG19 models in the classification of Parkinson’s atrophy from the brain MRI image dataset. The accuracy of the PD Net was 98%, whereas VGG19 and VGG16 models attained accuracy of 96%, and 91% respectively. Additionally, the sensitivity and specificity, for VGG19 were consistently higher than those of VGG16 and VGG19. In conclusion, the developed system proved to be efficacious in precisely categorizing Parkinson’s brain atrophy using the VGG19 classifier.