Oral cancer remains a critical global health challenge, characterized by high morbidity and mortality due to late-stage diagnosis. This paper addresses the need for improved diagnostic accuracy by introducing a novel hybrid-attention aided deep learning model for oral cancer classification. Our approach integrates the EfficientNetB0 model for feature extraction with a customized hybrid attention module, combining self, spatial, and soft attention mechanisms to emphasize crucial features. The model has been trained and evaluated on both microscopic and macroscopic oral cancer datasets, demonstrating its generalizability. Our proposed model achieves an accuracy of 96.97% on a macroscopic dataset and 92% on a microscopic dataset, with respective F1-scores of 0.97 and 0.89, outperforming baseline models. This hybrid attention-aided deep learning model shows significant potential in improving early detection and treatment planning of oral cancer. Source code and additional results of this work can be found at the GitHub repository.