Gestures, as a natural and intuitive form of communication, offer broad applicability and strong resistance to interference. This paper investigates a human–robot collaborative gesture recognition method. By leveraging the Mediapipe framework, accurate data is extracted to construct a gesture dataset. A multi-class gesture classification algorithm is implemented using the Support Vector Machine (SVM), and the Particle Swarm Optimization (PSO) is employed to optimize two critical SVM parameters. This approach improves overall gesture recognition accuracy by approximately 3%, with recognition rates for certain gestures increasing by up to 12%. The final method achieves an average gesture recognition accuracy of 97.33%. Through experiments simulating collaborative assembly scenarios in a laboratory environment, the proposed method was validated for its interaction accuracy, real-time performance, and overall rationality and effectiveness. This research enhances the resource efficiency, promotes energy conservation, and is important for improving sustainability.