Brain tumor classification from MRI scans demands precise image analysis, a challenge compounded by the variable morphology and location of tumors. Addressing this, our study presents an innovative approach that combines edge detection with a hierarchical deep learning framework to classify brain tumors accurately. This method enhances edge clarity, facilitating the deep learning model’s ability to distinguish between meningioma, glioma, and pituitary tumors. By deploying a two-stage model, initially segregating a meta-tumor class and pituitary and subsequently refining the meta-tumor class into glioma and meningioma with a binary classifier, we capitalize on the strengths of both traditional image processing and advanced neural networks. The already proven ResNet50 architecture, our model’s backbone, benefits from transfer learning, enabling efficient feature extraction from the edge image tailored to brain tumor recognition. Our results, evidenced by an over 96% overall accuracy rate obtained on a large benchmark brain tumor dataset, underscore the potential of integrating edge detection processing with deep learning. This integrative multi-level strategy promises to streamline the diagnostic process, offering a reliable, fast, and cost-effective solution that could reduce the need for expensive human specialist intervention.