The detection of brain cancers has always presented difficult and complex problems, which led to the development of sophisticated and new methods such as AI, ML, and DL for the analysis, segmentation, and characterization of brain MRI datasets. The novelty of this paper is that it performs detection, segmentation, and classification on the same dataset. In this paper, the first task is to perform detection, instance segmentation, and classification using YOLOv8 on the MRI dataset for brain cancer, which makes it simplify these tasks using YOLOv8. The second task is to propose a CNN model for brain cancer classification and compare it with the classification model of YOLOv8. The brain tumor dataset contains MRI images and annotation data to provide a bounding box for tumor recognition and characterization. Brain cancer detection, segmentation, and classification are achieved on the MRI dataset by the new version of YOLOv8. The results of numerous studies are presented using performance matrices, mainly mAP (mean average precision). The performance measures are achieved through the confusion matrix in classifying the MRI dataset using YOLOv8, which is lower than the proposed CNN model.