Dynamic bus scheduling plays a critical role in reducing operating costs and maintaining service quality amid real-world uncertainties such as traffic congestion and vehicle breakdowns. However, existing approaches often overlook two key aspects: (1) the lack of in-service vehicle information, limiting situational awareness and leading to suboptimal dispatching decisions; and (2) the absence of performance metrics that explicitly account for workload imbalance, resulting in uneven vehicle utilization and long-term inefficiencies. To address these limitations, we propose ERL-BSA, a reinforcement learning method enhanced by OpenAI Evolution Strategies. ERL-BSA features a dual-branch multi-head attention network that integrates both in-service and depot vehicle data, and a new reward function that jointly optimizes fleet size, deadhead trips, and workload balance. Experiments on three real-world datasets demonstrate that ERL-BSA significantly improves workload distribution while reducing fleet size, highlighting its practical value for smart transit systems.