Methods for detecting irregular spatial clusters encompass a wide range of practical applications, establishing themselves as valuable tools for analyzing disease outbreaks and other phenomena. However, spatial scan statistics, widely adopted as a methodology in these analyses, require the support of strategies to mitigate the overestimation of candidate clusters. To address this challenge, multi-objective optimization techniques have been introduced, which optimize the scan statistic simultaneously with a penalty function applied to the shape or structure of candidate clusters. We propose an innovative method based on a Multi-Objective Reinforcement Learning (MORL) paradigm with a specialized Multi-Objective Markov Decision Process (MOMDP). Our approach centers on a novel Pareto Q-Learning Scan (PQL-SCAN) algorithm that dynamically learns an efficient policy set. This method generates candidate clusters by optimizing a reward vector defined by two conflicting objectives: maximizing the spatial scan statistic and minimizing the dispersion penalty function. Comprehensive computational experiments were initially conducted on a synthetic dataset map with artificial clusters, followed by evaluations on a real-world disease map. The results demonstrate the high efficiency, robustness, and adaptability of the PQL-SCAN in accurately detecting complex irregular clusters.