<p>Plants are the primary source of food, and a secure food supply chain is essential for societal stability. Plant diseases and pests caused by seasons, environment, and pathogens can severely impact the yield of economic and food crops. Currently, farmers cannot obtain timely and effective information on the growth and distribution of diseases and pests during farmland management, leading to reduced crop yields and frequent pesticide use. Therefore, non-destructive, timely, and accurate methods for identifying diseases and pests are essential for promoting precise field management and increasing crop yields. The widespread application of artificial intelligence, particularly deep learning (DL), offers new opportunities for this approach. This paper reviews relevant literature published since 2010. It outlines the background of DL, commonly used disease and pest datasets, performance evaluation metrics, and data augmentation methods for both regular and small targets. It reviews the application status, challenges, and future research directions of DL in asymptomatic detection, visual detection and recognition, and small object detection of plant diseases and pests. The key advancements of DL in plant disease and pest recognition and detection will help further explore and expand its application in agriculture.</p>