Internet-wide scanning is essential for network measurement, topology discovery, and security research. While IPv4 scanning has significantly advanced these fields, the vast size of the IPv6 address space renders exhaustive probing infeasible. With the continued growth of IPv6 adoption, developing efficient scanning strategies is critical for comprehensive measurement. This paper presents 6Scout, an online IPv6 scanning framework that integrates a hybrid-tree partitioning model with reinforcement learning to maximize active-address discovery under constrained probing budgets. 6Scout first extracts structural features from known active seeds and clusters them into high-density regions. An Empirical Cumulative Distribution-based Outlier Detection (ECOD) filter is then applied to remove outlier seeds, reducing scanning space while preserving coverage. Finally, the framework formulates region-level scheduling as a Multi-Armed Bandit problem and employs an Upper Confidence Bound (UCB) strategy to adaptively allocate probes toward regions with the highest estimated yield. Experiments on multiple real-world IPv6 datasets show that, under equal probing budgets, 6Scout improves effective target density by 135%–233% and hit rate by 15%–100% compared with state-of-the-art baselines, demonstrating strong robustness and adaptability across diverse network environments.