Brain cancer is a highly dangerous disease because symptoms do not appear until the disease has already spread to other areas of the body. Thus, early detection and classification of abnormal severity is highly important to reduce mortality due to brain cancer. In this work, the classification of brain tumor is performed using advanced computational approaches that analyze digital medical imaging. Specifically, it identifies whether a brain tumor exists and, if so, characterizes the severity by one of four grades such as mild impairment, moderate impairment, very mild impairment, and no impairment. Toward these aims, two of the latest deep learning architectures such as BT-SqueezeNet (Brain Tumor- SqueezeNet) and BT-ShuffleNet (Brain Tumor-ShuffleNet) are proposed in this study to classify the brain tumor. These models are tuned to yield high accuracy and robustness with little computation. SqueezeNet acts as the primary framework because of its fewer computation parameters; however, the performance of the architecture can be improved further by introducing BT-ShuffleNet. The overall classification accuracy of 96.17% is achieved with the proposed BT-ShuffleNet architecture.