<p>Gastrointestinal (GI) tract image segmentation is an important stage in automated diagnostic systems. Segmentation can be assistive and provide information about which area to be segregated. However, the challenges, such as low contrast, irregular shapes, and fuzzy boundaries, make this process challenging. This article presents a lightweight attention-based framework for polyp segmentation. The proposed framework utilizes the EfficientNet-b0 in the encoding phase. A custom decoder is developed that integrates the channel-spatial attention and dynamic gating modules. The attention module enhanced the feature representation, and the dynamic gating module adaptively selects and fuses encoder–decoder features. The auxiliary deep-supervision heads are integrated at the intermediate decoder phases to refine features and stabilize learning. The decoder output is provided to two prediction heads for boundary and mask prediction. A hybrid loss is designed that combines Dice, Lovász-hinge, and Binary Cross Entropy (BCE) loss to optimize region overlap, contour precision, and handle imbalanced classes. For interpretability, Gradient-weighted Class Activation Mapping (Grad-CAM) is incorporated into the proposed framework. The proposed framework provides 94.92% Dice and 90.85% Intersection over Union (IoU) on the Kvasir-Seg, 94.00% Dice and 89.74% IoU on the Hyper-Kvasir, and 93.99% Dice and 88.84% IoU on the CVC-ColonDB dataset. Results on the proposed framework demonstrate significant performance in segmenting the polyps in GI tract images.</p>