In optimization, overcoming the challenge of local optima stagnation is crucial due to its propensity for converging towards suboptimal solutions rather than achieving globally optimal ones. This study introduces an experimental approach that employs Shannon entropy as an analytical tool for detecting stagnations at local optima, combined with the Q-learning technique to enhance particle position updates. We applied this approach to the NP-hard Dial-a-Ride Problem, which is a complex problem in modern engineering. Its relevance and applicability suggest a promising framework for future research in optimizing bio-inspired algorithms beyond this specific context. While preliminary results suggest the modified Particle Swarm Optimization model does not consistently outperform the standard method in every aspect, it demonstrates significant effectiveness in detecting and resolving stagnations.