Deep Reinforcement Learning Based Dynamic Bus Timetable Scheduling with Bidirectional Constraints

Bus timetable scheduling is a vital issue in reducing the operational costs of bus companies and improving service quality. Existing methods schedule the timetable only for one direction of the bus line, which may lead to the issue of inconsistent departure numbers for the upward and downward directions on the same bus line. For bus companies, it is very important to have an equal number of departures in both directions. To ensure each bus can return to its original departure point, subsequent bus schedules should be based on a timetable that ensures an equal number of departures in both directions. This study proposes a Deep Reinforcement Learning-based dynamic bus Timetable Scheduling method with Bidirectional Constraints (DRL-TSBC). In DRL-TSBC, the problem of bus timetable scheduling in both directions is formulated as a Markov Decision Process (MDP). A Deep Q-Network (DQN) is applied to determine whether to depart in both directions every minute. We construct a state that includes information on bus lines in both directions. Considering the need for consistency in the number of departures in both directions, we design a reward function to ensure an equal number of departures in both directions, while also balancing the bus company’s costs with the passengers’ experience. Through experiments, we find that DRL-TSBC can effectively reduce the average waiting time for passengers compared to the scheme currently used in reality. At the same time, when dealing with dynamic passenger flow changes, DRL-TSBC can make adjustments to adapt to the changes in passenger flow.