In the field of medical imaging. While convolution-based networks have demonstrated impressive results in medical picture segmentation, they are constrained in their ability to simulate extensive contextual relationships and spatial Tumors such as necrosis, edema, and enhancing tumor exhibit significant similarity in features on multimodal MRI. These homogeneous features include magnetic resonance imaging. These similarities provide challenges in dependencies. The Pyramidal convolution transformer has exhibited exceptional proficiency in diverse workflows through the utilization of self-supervised learning, owing to its influential capacity for long-range information interaction. This paper introduces PAT-Unet, a pyramidal attention transformer-based network designed for brain tumor segmentation. The motivation behind this work stems from the vision transformer. During the tumor classification phase, the PAT-Unet generates multiscale features that serve as input for sub-region categorization. To increase the segmentation performance, the fine-tuning (FT) phase then combines and refines these characteristics. Furthermore, we introduced a self-supervised learning architecture for pre-training the encoder of the model by means of the reconstruction task. Our suggested model underwent thorough experimentation on tumor segmentation tasks, and the results consistently showed superior performance with 94.6% dice-score average, surpassing established standards.