This paper systematically investigates the effects of different lighting conditions on broccoli head recognition and segmentation in the field environment for a broccoli harvesting robot. First, an image data acquisition system was designed and set up for different lighting conditions in the field, and a dataset was collected under different natural lighting conditions and shading conditions with varying auxiliary light intensities. Based on the YOLOv8 object detection and Segment Anything Model(SAM) image segmentation models, the impact of different lighting conditions on broccoli recognition and segmentation was analyzed. The results indicate that under oblique sunlight conditions, the broccoli recognition rate is 91.2%. After sunset, the recognition rate of broccoli head based on YOLOv8n decreases with the decrease of light intensity, and the recognition rate is 95% when the light intensity is 83.33 lx. The effectiveness of shading during the day affects the relative effectiveness of artificial auxiliary light sources. When shading is effective, with light intensity increasing from 400 lx to 1800 lx, the recognition rate remains around 95%. At night, the recognition rate remains around 98% as shading and the intensity of artificial auxiliary light sources increase from 200 lx to 1800 lx. Under shading and additional auxiliary light conditions, the impact of different light intensities on SAM’s broccoli head segmentation performance is minimal, with an average effect on diameter of only about 3.33%. This study provides a reference for the subsequent design of visual systems and optimization of broccoli head recognition algorithms for broccoli harvesting robots.