This work explores the improvements and methods that led to SinfonIA Uniandes’ success in winning the RoboCup@Home Social Standard Platform League (SSPL) 2024, using the Pepper robot from SoftBank Robotics. Considering the challenges due to human-robot interaction (HRI) and the hardware constraints, our focus was on improving Pepper’s ability to offer practical assistance in home tasks. Key improvements include a robust person-following system that combined YOLO-based visual detection with a PID controller, achieving 80% accuracy in tracking individuals during short walks (under one minute). To improve General Purpose Service Robot (GPSR) tasks and perception capabilities we integrated OpenAI’s GPT-4o, significantly enhancing Pepper’s interaction and functionality. This enabled Pepper to do more complex operations like determining whether a person is wearing shoes or interpreting gestures with an 89% success rate in executing autogenerated GPSR tasks. The integration of GPT-4o with OpenAI’s locally deployed Whisper system provided robust speech recognition, facilitating smoother and more natural conversations, crucial for complex human-robot interactions. Additionally, GPT-4o’s multimodal capabilities enabled advanced image processing, enhancing the robot’s understanding of its surroundings. Our work with Pepper underscores the potential of AI to address key hardware and perception challenges, empowering social robots to effectively assist humans in dynamic environments.