Capacitated Vehicle Routing Problems with Time Windows (CVRPTW) pose significant challenges due to their complex temporal and capacity constraints. In this paper, we propose a novel two-phase framework that leverages deep reinforcement learning and attention mechanisms to efficiently decompose and solve CVRPTW instances. In the first phase, we employ an attention-based model trained with reinforcement learning to partition a given CVRPTW instance into a set of feasible sub-problems formulated as Time-Windowed Traveling Salesman Problems (TSPTW). Our model is guided by a rich context embedding that integrates both local sub-route structures and global graph-level information, enabling high-quality, constraint-aware partitioning. In the second phase, each TSPTW sub-problem is solved using a specialized solver, allowing parallelization and efficient handling of time window constraints. Experimental results demonstrate that our approach achieves competitive performance in terms of solution quality and scalability, especially on large instances where classical methods struggle. This work highlights the potential of hybrid learning-decomposition strategies for addressing real-world logistics optimization tasks.