Satellites have become an essential part of our daily lives, from communication systems and satellite television to remote sensing and data collection. However, a single malfunction in these multi-billion-dollar satellites leads to their decommissioning and retirement, resulting in the loss of essential services. To address this challenge, robotic systems are being integrated into On-Orbit Servicing missions for satellite repair, as well as the assembly and maintenance of large space infrastructures. While robotic systems are physically tested before deployment, these tests are often limited in scope and cost-prohibitive. Reinforcement Learning (RL) techniques implemented through simulation environments can offer a cost-effective approach that accelerates training and enhances the capabilities of these systems in complex space environments. This paper introduces AstroArm, an innovative, open-source simulation environment to enhance robotic system capabilties in satellite servicing through an RL training pipeline. Currently, there are no publicly available, free simulation environments to train robotic arms using multiple RL methodologies for complex satellite repair tasks. AstroArm fills this gap, incorporating RL algorithms such as the REINFORCE and Actor-Critic Proximal Policy Optimization methods to offer cost-effective, accelerated training for satellite servicing. We demonstrate AstroArm for a robotic hand required to perform the complex task of removing the Science Instrument Control and Data Handling unit from the Hubble Space Telescope. We compare AstroArm with existing environments, highlighting its unique capabilities for advancing AI-driven space robotics.