Driver Safety System for Agricultural Machinery Operations Using Deep Learning Algorithm

One of the most critical aspects of ensuring the safety of agricultural machinery drivers is the timely recognition and detection of dangerous actions of the driver for subsequent safe handling. This research aims to identify and classify the driver’s actions while driving to determine whether the driver is in dangerous behavior and thus be able to raise an early warning. Considering that there are few studies and datasets in agricultural driving security systems, a dataset on agricultural driving is needed to assist people in this area of research. The driver action dataset collected in this research includes five categories: looking around, talking, playing on the phone, drowsiness, and normal driving. In this research, offline and online training validation methods were designed to evaluate the classification performance of video classification algorithms by a modified MoViNet model. Considering that the safety system requires very high real-time performance, this research replaced all 3D convolutions of the model with depth-separable convolutions to obtain the contextual semantic information of the temporal layer and spatial dimensional information, and CGAP obtained the temporal layer. The SE block obtained spatial dimensional information. The model had 70% offline classification accuracy, 70% regression, 70% accuracy, 0.70 F1 scores, and 0.90 AUC scores on the validation set. The offline classification accuracy of the model on the test set was 80%, the recall rate was 79%, the prediction was 81%, the F1 score was 0.79, and the AUC score was 0.93. At the same time, the model could achieve 10 fps when inferring online, and the recognition effect did not decrease, basically realizing real-time online processing. In conclusion, this research contributed to developing agricultural driving safety systems and facilitated the researchers’ study in the same field.