Mechanized tunneling projects are complex systems influenced by numerous variables. For efficient project implementation, not only core processes of the tunnel boring machine (TBM), but logistical processes of material supply and disposal must be considered as supporting processes. Poor logistics planning can cause bottlenecks, increased costs, and delays. Simulation techniques can be used to model and analyze logistics both in the planning phase and during implementation to identify potential bottlenecks and performance-influencing factors at an early stage and enable a transparent evaluation of selected logistics strategies or project configurations. This paper presents a concept that uses reinforcement learning (RL) to continuously improve logistics processes in mechanized tunneling projects through a simulation-based approach. RL algorithms enable dynamic adaptation of logistics systems to changing conditions in the simulation environment and support decision-making through adaptive learning. The focus is on external logistics, including supply and disposal of construction sites. The concept aims to show how RL can be used to minimize time-consuming processes and address factors of cost efficiency and sustainability in external logistics. Logistical setups are analyzed through generic implementation to learn from an environment with an RL agent and identify logistical strategies that ensure smooth supply and disposal of construction sites.