In this work, we present a scalable approach for the automatic detections and segmentation of fruits, flowers and diseased leaves in RGB images. While many general methods for detection and segmentation demonstrate high performance, they underperform in domain-specific scenarios as they have not been trained for specific and fine-grain applications like the ones we address here. General-purpose methods can be adapted to specific tasks if large, annotated datasets are available for training. However, creating such data is labor-intensive, requiring capturing plants under a variety of conditions as well as domain specific knowledge for image annotation. To address this challenge, we propose a methodology that combines fine-tuning of general-purpose vision models with efficient annotation techniques, including pseudo-labeling and data augmentation. Evaluation using COCO API metrics demonstrates high detection and segmentation accuracy with reduced annotation overhead. Our approach offers a scalable and efficient solution for automated plant monitoring, supporting early disease detection, resource optimization, and improved crop health assessment in precision agriculture.