5G cellular networks must serve diverse user equipment (UEs) with heterogeneous Quality-of-Service (QoS) requirements. Traditional packet schedulers optimize throughput, making them ineffective in delay-sensitive scenarios. Deep reinforcement learning (DRL) offers a promising approach to QoS-aware scheduling but faces two challenges: (1) direct decision-making methods suffer from a large discrete action space due to numerous resource blocks (RBs), and (2) indirect methods tuning heuristic parameters focus on optimizing throughput without considering delay constraints. To address these issues, we propose a novel indirect DRL-based scheduling algorithm that integrates a priority-based heuristic for RB allocation, with DRL optimizing its parameters. To enforce delay constraints, we introduce a priority metric incorporating delay events and redesign the priority formula with tunable parameters to balance delay sensitivity and throughput. We employ Twin Delayed Deep Deterministic Policy Gradient for continuous parameter control, Deep Q-Network for discrete parameter control, and a Graph Convolutional Network to handle varying UE counts. Simulations on a realistic 5G NS3 simulator show that our approach outperforms state-of-the-art methods by ensuring QoS satisfaction while maintaining competitive throughput.