Memory impairment and cognitive decline are hallmarks of Alzheimer’s disease (AD), a degenerative neurological disorder. For prompt action to limit its course, an early and accurate diagnosis is essential. Recently, multimodal data integration has emerged as a promising strategy to increase the accuracy of AD diagnoses. These techniques lower the danger of overfitting and increase diagnostic accuracy by integrating data from many sources. High-dimensional multimodal data, however, requires a lot of computing power. Small feature sets are extracted to improve model generalization and lower dimensionality in order to address this. We describe a multimodal small feature set-based approach to AD diagnosis in this study. We provide a method that combines between-group difference analysis with a multilayered attention mechanism. Using intergroup difference analysis, we filtered 114 SNPs, 77 RGV features, and 18 sMRI brain areas to create a low-redundancy small feature set. To create an AD diagnosis model using a multimodal small feature set, we integrated a convolutional neural network (CNN), a multilayer perceptron (MLP), and a multilayer attentional mechanism. 96.14% was the model’s average accuracy rate. Furthermore, by identifying important brain regions and genetic markers associated with AD, this approach provides insights into the disease’s genesis. Our strategy beats numerous modern AD diagnostic techniques, even if diagnostic performance based on multimodal small feature sets would not be as good as models employing entire data. Critical diagnostic information is retained while data dimensionality is efficiently reduced using the multimodal small feature set-based technique. It enhances computational efficiency and model interpretability, providing new avenues for neurological disease early detection.