Research on Worker Action Recognition and Evaluation in Intelligent Manufacturing Training Based on Industrial Metaverse

Industrial Metaverse plays an increasingly important role in Intelligent Manufacturing Training, improving the user immersion as well as reducing the cost and risks in training. Action recognition technology can effectively detect human actions, help construct the industrial metaverse space of Human-in-the-loop, and empower the low-intrusion, personalized intelligent manufacturing training system. Intelligent Manufacturing Training-Worker Action Recognition and Evaluation (IMTware) is researched to improve the human-manchine interaction and intelligent guidance in industrial training. In terms of action recognition, IMTware extracts the skeletal point sequences of human practice training actions in the video via OpenPose network and inputs them into ST-GCN network for action recognition. In terms of action evaluation, a method combining Dynamic Time Warping (DTW) keyframe matching with cosine similarity solving is proposed to compute the action similarity between the skeletal point sequences and the standard skeletal point sequences, and evaluate them according to the similarity to give personalized correction suggestions. Subsequently, the performance of the system is validated using overhead crane command actions training in converter steelmaking as an example, achieving satisfactory results.