In order to solve the problem of real-time detection of tomato quantity, location and maturity, a real-time tomato detection scheme based on FPGA was designed. The scheme uses the quantizer and compiler in the Vitis-AI tool to quantize and compile the YOLOv5s algorithmic model respectively, obtains the optimized model and deploys it to the FPGA accelerator platform, and then uses the ZCU104 FPGA accelerator to accelerate the inference of the YOLOv5s algorithmic model, so as to be able to realize the real-time detection task of tomato on the hardware platform. The experimental results show that the recognition effect of the YOLOv5s network in this paper has a certain improvement over the comparison YOLOv5s network in terms of precision rate, recall rate, and mean average precision rate.