Human activity recognition is crucial in surveillance systems as it allows for accurate monitoring and interpretation of human behaviors in different environments. This study presents an improved method for activity recognition by combining multi-person pose estimation with advanced machine learning techniques. By using the YOLOv8 pose model, we can accurately detect and extract 17 keypoints from each individual, representing essential body joints in both visible and infrared images. These keypoint coordinates are then analyzed by an ensemble classification system, which integrates K-nearest neighbors (KNN), support vector machine (SVM), random forest, and naive Bayes models using soft voting. This ensemble approach effectively classifies activities such as standing, sitting, crawling, walking, running, jumping, kicking, and punching. The YOLOv8 pose model achieves a keypoint detection accuracy of 90.0%, while the ensemble classifier attains an activity classification accuracy of 93.0%. This approach demonstrates the effectiveness of combining pose estimation with ensemble learning to enhance activity recognition, providing reliable performance in diverse surveillance conditions.