5G deployments highlight the need for efficient infrastructure optimization. As networks become more complex, traditional algorithms struggle, requiring AI and ML approaches. This paper presents a deep Q-Network (DQN)-based agent for optimal Physical Resource Blocks (PRBs) allocation, focusing on energy saving while ensuring service requirements through interference mitigation and resource allocation. The agent employs a multi-stage reward algorithm to achieve distinct sub-goals and a training process that enables generalizable optimization across all cells in the scenario. The results demonstrate that this design enables the agent to efficiently optimize multiple cells in a coordinated manner.