The perception module is kind of the “eyes” for an autonomous vehicle. It directly connects with sensors, obtains a large amount of input data from sensors, and then obtains an understanding of the surrounding environment after analysis and processing. Section 5.1 introduces the calibration of various on-board sensors; Sect. 5.2 introduces various methods of distance measurement with a single camera; Sect. 5.3 introduces the depth estimation algorithm of a single image; Sect. 5.4 analyzes the current 3D obstacle detection method of a single camera; Sect. 5.5 discusses obstacle tracking methods; Sect. 5.6 analyzes the fusion methods from the two levels of data and task; Sects. 5.7 to [2] mainly discuss three issues of concern to the on-board vision system: lane line detection, traffic sign detection, and traffic signal light recognition; Sect. 5.10 focuses on the issue of the drivable area seen by the visual camera; Sect. 5.11 is an analysis of the perception module based on stereo vision. Due to the need for pedestrian behavior analysis, human posture estimation is discussed in Sect. 5.12; Sect. 5.13 specifically mentions the implementation algorithm of the Driver Monitoring System (DMS); Sect. 5.14 deeply discusses the current latest BEV (bird’s-eye view) perception system; and Sect. 5.15 analyzes the extension of the BEV method: occupancy network.