Current AI education concepts often emphasize supervised machine learning algorithms, while reinforcement learning (RL) education remains limited, typically focusing on introductory concepts. To address the need of a more in-depth RL educational experience, we propose two hands-on robotic activities to introduce foundational RL concepts and foster exploration of RL system design principles. These activities include a LEGO® car task and a beam-and-ball balance robot simulation. We conducted a pilot study featuring this two-part curriculum with 13- to 16- year old students and analyzed their learning outcomes quantitatively. Our findings indicate that middle and high school students developed an understanding of basic RL concepts as measured by post-activity reflections. We also discuss next steps to enhance the curriculum, including providing a more interactive experience with RL system design.