Diseases affecting the gastrointestinal system (GIS) have witnessed a surge in the past decade, attributed to significant lifestyle changes. Gastrointestinal (GI) diseases encompass a wide range of conditions affecting the digestive tract, from mouth to anus. These diseases can result from various causes such as infections, inflammation, dietary factors, genetic predisposition, and lifestyle choices. This research introduces a transfer learning-based ensemble approach in response to the escalating challenge of manual analysis of gastrointestinal tract images, particularly from wireless capsule endoscopy or video capsule endoscopy. Utilizing a transfer learning approach with adjusted fully connected and output layers, we achieved individual accuracy rates of 96.46%, 95.94%, and 95% with deep learning models such as ResNet50, ConvNextBase, and EfficientV2M, respectively. Ensemble techniques such as model averaging and weighted averaging were then employed, resulting in outstanding accuracies of 96.88% and 98.0%, surpassing existing state-of-the-art models.