Performance Evaluation of Deep Transfer Learning and Semantic Segmentation Models for Crop and Weed Detection in the Sesame Production System

India’s agriculture sector brings in over $375 billion annually. Regarding the production of agricultural goods, India comes in second. With the help of precision agriculture, the yield can be significantly increased. In precision agriculture, crop, and weed detection is among the most critical issues. The robotic weeding technique can be used for weed management. So precise and tailored weed treatment is possible in such a system by correctly identifying weeds and crops. Despite recent advancements, reliable crop and weed identification and localization in the unstructured field is a significant issue that necessitates supervised modeling with annotated data. This paper examines the performance evaluation of Deep transfer learning (DTL) and pixel-based semantic segmentation models for crop and weed identification in Sesame production system. The publicly available dataset of Sesame and its weeds are used and modified for experimentation. After modification dataset contains 7721, 10298, and 35350 images for Weeds, Crops, and Background. 21 Deep learning models through transfer learning and 18 pixel-based semantic segmentation models are used to identify crops and weeds using real images. With and without augmentation, MobileNet has the highest accuracy, 94.73% and 96.67%, respectively. After image augmentation, MobileNet’s accuracy is improved by 1.94%. ResNet50, however, exhibits the most remarkable change in accuracy following augmentation, i.e., 5.97%. Across 18 semantic segmentation models, Unet Mini has the most significant Frequency Weighted Intersection over Union and Mean Intersection over Union in the semantic segmentation model, at 94.34% and 78.06%, respectively.