The measurement of surface deformation of asphalt pavement and the acquisition of rut shape are of great significance to pavement management agencies. Currently, the high cost and maintenance conditions of rut measurement systems make it difficult for road management departments in developing countries to obtain accurate rut conditions in the region. Therefore, a cost-effective system for the collection and analysis of asphalt pavement rut data based on RGBD sensors is proposed. Firstly, the data collection method is presented, integrating the RGBD sensor into the rut measurement system. Then, methods for point cloud processing and rut condition index calculation are proposed. Finally, the system accurately measures rut width and maximum depth, and precisely locates the position of the rut within the lane by employing an adaptive depth value watershed algorithm combined with continuous precise measurement of rut characteristics. This study conducts field experiments to evaluate the capabilities and limitations of the proposed system. The results indicate that the measurements obtained using the new algorithm are highly correlated with the ground truth collected manually on-site. This research assists transportation agencies in conducting traffic safety analysis, ride comfort assessment, and structural damage prediction based on continuous pavement rut dimensions and positions.