Early and accurate breast cancer detection is vital for timely treatment and improving chances of patient recovery. There have been comprehensive studies going on for decades to identify patients with breast cancer using novel machine learning techniques. Traditional methods like convolutional neural networks have limitations and are not able to capture long-range dependencies. The latest developments in transformer architectures have successfully addressed this issue, leading to improved performance in various computer vision tasks, including image classification and semantic segmentation. This project investigates the performance of shifted window transformer (SWIN), neighbourhood attention transformer (NAT) and vision transformer (ViT) architectures for breast cancer detection using the challenging BreakHis dataset. It also compares the performance of these transformers with industry leading convolutional neural networks (CNNs) architectures like ResNet50, VGG16, EfficientNetB1. We propose a comparative framework for these advanced transformers in our project. SWIN and ViT utilise attention mechanisms to capture long-range dependencies, while NAT uses a hybrid approach with local convolutional features and global transformer attentions. The study employs data preparation and fine-tuning techniques to optimise model performance.