This study presents a comprehensive comparison of eight deep learning architectures for brain tumor classification using MRI scans. We evaluate seven transfer learning models and a Vision Transformer (ViT) on both binary and multi-class tumor classification tasks across two publicly available datasets (Br35h and Sartaj). Our findings reveal that ViT achieves superior performance in binary classification, while ResNet101V2 excels in multi-class differentiation. We observe that lightweight architectures like MobileNet deliver comparable performance to more complex models, suggesting their viability for resource-constrained clinical settings. The performance difference between binary detection and multi-class differentiation highlights persistent challenges in distinguishing between specific tumor types. This comparative analysis provides guidance for selecting appropriate architectures for brain tumor classification based on specific clinical requirements, computational constraints, and classification objectives. Our results contribute to the ongoing advancement of AI-assisted diagnostic tools for neurological disorders while identifying key directions for future research, including model interpretability, volumetric analysis, and federated learning approaches.