Adaptive queue management in healthcare using supervised Q-learning with time-varying reward and cost structures

Efficient patient management in hospitals requires adaptive decision-making under time-varying demand and dynamic service environments. This study proposes a heterogeneous medical patient queueing model that integrates reinforcement learning with stochastic queue dynamics to minimize overall patient waiting time. The model distinguishes between two categories of service providers (SPs): those attending first-time patients and those serving returning patients. Each category may differ in service rate but not in medical specialty. Patient arrivals follow a non-homogeneous Poisson process (NHPP) to capture realistic time-dependent flow variations. A Q-learning framework with a supervised \(\epsilon\)-greedy policy is developed to determine optimal operational actions, such as adding or reallocating service providers, based on system state and event type. Separate Q-tables are maintained for arrival and departure events to account for differing cost and reward dynamics. Simulation results demonstrate that the proposed model significantly reduces total waiting time and system cost compared with conventional homogeneous queue models. This approach provides a data-driven mechanism for dynamic hospital queue management and can be extended to broader healthcare resource optimization scenarios.