Efficient resource allocation is critical for reducing the mean cycle time in business processes. While traditional heuristics like Shortest Processing Time (SPT) and First-In-First-Out (FIFO) are widely used, their effectiveness depends heavily on process characteristics. This paper introduces a Deep Reinforcement Learning (DRL) approach that dynamically selects heuristics during process execution. Our method learns adaptive heuristic selection, automatically determining when each heuristic should be applied to minimize the overall cycle time based on the current process state. Furthermore, unlike existing methods that encode each resource-to-activity assignment as a separate action, our approach limits the actions to the considered heuristics, reducing the dimensionality and complexity of the learning task. We evaluated our method on six synthetic and five real-world business processes. Our proposed method outperformed the best individual heuristic in six out of eleven scenarios and matched performance in the remaining five. The results demonstrate that adaptive heuristic selection using DRL provides a scalable and effective strategy for resource allocation that adapts to varying business process characteristics.