Early diagnosis of gastrointestinal (GI) conditions through endoscopy can drastically improve patient outcomes. However, the accuracy and consistency of endoscopic interpretation often vary with operator expertise, and existing AI-assisted solutions remain too resource-intensive for deployment on low-powered clinical devices. In this work, we propose a lightweight, human-in-the-loop (HITL) active learning framework that incrementally fine-tunes a compact deep learning model based on expert feedback collected during clinical deployment. Initially trained on a limited subset of annotated endoscopy images, our model is optimized for real-time inference on edge hardware (Jetson Nano). During deployment, expert feedback on erroneous predictions is collected and used for periodic fine-tuning without increasing model complexity. Our system supports continual improvement, maintains fixed model size, and demonstrates enhanced diagnostic performance in both sensitivity and precision, thus offering a robust, adaptive solution for AI-assisted endoscopic diagnostics in resource-constrained environments.