Social Robots pose an engaging alternative to support physical stimulation exercises, complementing the gamification of custom-designed activities. This paper presents the implementation of a music generation system integrated into a multi-agent framework aimed at stimulating upper limb movements while the user plays a ludic activity based on a musical game that includes different game modes like free mode, drum pad machine mode, memorizing the song, or following the song. The proposed system has been integrated into the software architecture of the Mini robot, which a game agent accompanies. Mini guides the activity to promote the user’s physical activity. The game agent is responsible for tracking the user’s hand movements using a hand detector to control the game. It facilitates some metrics for evaluating physical stimulation, such as the user’s chosen playing hand, gameplay duration, hand movement frequency, and game scores. Our approach can complement conventional therapies by making use of social robotics.