Public speaking anxiety (glossophobia) is a widespread issue that can impact professional development. While Virtual Reality (VR) has emerged as a promising tool for exposure-based public speaking training, many existing systems lack features such as real-time physiological feedback, adaptive responses, and engaging gamification elements. This paper presents a VR-based system to address these gaps by integrating heart rate monitoring, adaptive feedback mechanisms, and gamified components to improve user engagement and reduce anxiety. The system also supports scalable virtual environments, enabling users to practice in settings that simulate increasing social pressure. A user study with 15 participants was conducted to evaluate the system. Subjective measures included questionnaires on public speaking anxiety (PRPSA), usability (SUS), and immersion (Presence), while objective data included physiological signals, gaze behavior, and speech performance. Results show that, relative to the non-gamified condition, the gamified VR condition produced significantly lower self-reported anxiety, faster heart rate recovery, more balanced eye contact, and fewer filler words. The average PRPSA score improved by 16.4 points, and the system achieved a mean SUS score of 82.5. Overall, the findings demonstrate that combining physiological monitoring, gamification, and adaptive feedback within VR environments can offer an effective and engaging solution for managing glossophobia and enhancing public speaking skills.