The precise classification of gastrointestinal (GI) disorders is essential for appropriate diagnosis and therapy planning. This research introduces an innovative deep learning model founded on EfficientNet, enhanced with Squeeze-and-Excitation blocks and Self-Attention processes, aimed at tackling issues like inter-class variability, imaging artifacts, and inconsistent lighting in gastrointestinal endoscopy. The suggested architecture improves feature extraction and spatial contextual comprehension, demonstrating strong performance across four gastrointestinal conditions: Diverticulosis, Neoplasm, Peritonitis, and Ureters. Thorough assessments on a selected dataset reveal an overall accuracy of 95.43%, accompanied with balanced precision, recall, and F1-scores across all categories. Comparative assessments demonstrate substantial advancements over baseline topologies, validating the efficacy of the proposed upgrades. The model demonstrates state-of-the-art performance while also identifying areas for enhancement to facilitate real-time deployment and enhance clinical applicability. This study enhances AI-based diagnostics in medical imaging, highlighting the transformational potential of specialized architectural advances.