Reinforcement Learning (RL) is a classical machine learning technique that has seen an increase in use and research during the last years. It has found success in complex tasks, where understanding the relationship between the actions done and their consequences is key for fruition, in fields as diverse as robotics, videogames, autonomous systems and logistics. Nonetheless, RL is a deep area with a lot of terrain still in need of exploration. Past research shows how the design chosen for the environment has a heavy impact on the model outputs, and, therefore, the present study focuses on studying the impact the action space has over solution quality in a RL environment. With this aim, a RL environment for the Vehicle Routing Problem (VRP) has been implemented, where the influence of the action space has been analyzed through an experimentation process. The results validate the formulated hypothesis and leave room for further studies on the subject.