This paper introduces a novel automated decision-making system for integrating autonomous Mobility-on-Demand (AMOD) services with conventional public transport systems, focusing on two main optimization tasks: vehicle order matching (VOM) and vehicle relocation (RE). In VOM, the system decides which active orders are serviced by AMOD vehicles, assigns idle vehicles to these orders and decides which direct or combined routes with existing public transport should be executed. RE focuses on moving idle vehicles to better locations to boost network efficiency and ensure vehicles are optimally positioned for present and upcoming needs. Implemented in a Reinforcement Learning framework, this paper compares Q-Learning (QL) and Deep Reinforcement Learning (DRL) approaches to enhance operational efficiency in urban transport. The evaluation, conducted with real-world data from New York City, demonstrates that Reinforcement Learning significantly outperforms automated non-learning approaches, highlighting its suitability for enhancing AMOD services and their integration with existing public transportation systems.