This study addresses the challenge of optimizing ferry dispatch operations under uncertain and time-varying passenger demand by comparing a traditional static scheduling policy with a fully adaptive Q-learning (QRLP) approach that integrates online Bayesian estimation of arrival parameters. The QRLP agent dynamically adapts its dispatch policy using experience replay, while strictly enforcing operational constraints such as a minimum interval between departures and a maximum number of departures per hour. Simulation results demonstrate that the QRLP policy achieves significant reductions in total cost, passenger wait times, and lost customers compared to static scheduling. Sensitivity analyses confirm the robustness of these improvements across varying operational scales. These findings further highlight the practical value of integrating learning and control for real-world transportation systems.