Clock Drawing Test (CDT) is a widely used and clinically validated tool for cognitive assessment, especially effective in identifying early signs of cognitive impairment. However, manual scoring remains time-consuming and subjective, limiting its scalability in community screening. To address this challenge, this study proposes a deep learning-based cognitive impairment classification framework that integrates Convolutional Block Attention Modules (CBAM) into a customized residual network. The dual-branch attention mechanism enables the model to focus more effectively on cognitively salient features in CDT images, such as digit alignment, spatial layout, and clock hand positioning. The model is trained on the NHATS dataset containing 47,000 annotated CDT images and achieves six-level cognitive impairment classification (0–5 points) with an accuracy of 74.8%, achieving a 3.8% point improvement over baseline ResNet. The model achieves precision of 0.753, recall of 0.748 and F1-score of 0.749. Comparative experiments validate our approach’s competitive performance on a substantially larger dataset than existing methods. Furthermore, we demonstrate the clinical interpretability through systematic correlation between attention maps and specific drawing errors. This approach offers a scalable and objective solution for automated CDT analysis, with strong potential for real-world deployment in early dementia screening and large-scale cognitive assessment.