Recognition of miner action and violation behavior based on the ANODE-GCN model

In dimly lit coal mine environments, the accuracy of skeleton-based action recognition algorithms is frequently constrained by the absence of critical key points on the human body. To address this challenge, this paper proposes a miner action recognition model, termed ANODE-GCN, which integrates neural ordinary differential equations (ODEs) with graph convolutional networks (GCNs). In terms of feature extraction, both spatial and channel information are meticulously considered. A graph convolutional network equipped with a hybrid attention mechanism is designed to enhance the model’s ability to accurately differentiate between similar actions. Improvements to the network architecture are implemented based on the Info-GCN++ model structure. By utilizing neural ordinary differential equations, future motion sequences are predicted and combined with the current local motion states, thereby augmenting the robustness of action recognition in the context of incomplete skeletal data. For the recognition of violations, a novel strategy based on two-tuples is introduced, which matches the recognized miners’ fundamental actions with relevant environmental information, facilitating the accurate identification of more complex violation behaviors. Experimental results, derived from both publicly available and self-constructed datasets, demonstrate that the ANODE-GCN model significantly surpasses the original Info-GCN++ model in recognizing actions from incomplete skeletal data. Specifically, the Area Under the Curve (AUC) metric improved by 16.73% on the NTU RGB+D120 (X-Sub) dataset, reaching an AUC of 71.84%, and by 7.85% on the self-constructed dataset, achieving an AUC of 83.92%. Furthermore, in the task of violation behavior recognition, the average accuracy increased by 6.7%, reaching 91.7%, thus validating the model’s effectiveness and reliability within underground coal mine environments. The code for ANODE-GCN is publicly available at https://github.com/hou-sudo/ANODE-GCN for further exploration and validation.