This paper presents a novel system for potato blemish detection that integrates interactive image segmentation with a human-in-the-loop mechanism. By employing superpixel-based over-segmentation and an AdaBoost classifier, the method requires only a small amount of training data while achieving near-real-time performance. Experimental results on white and red potato datasets demonstrate detection accuracies of up to 93.7% for blemished regions, with classification times under 2 seconds per image. In contrast to traditional offline training methods, our approach facilitates rapid adaptation and offers a promising alternative to few-shot learning with pre-trained models. The system targets a range of defects, including black dot, scab, silver scurf, and greening, making it highly relevant for real-world food quality inspection applications.