Ultra-Reliable and Low Latency Communications (URLLC) is crucial for enabling next-generation applications, particularly in areas that require stringent delay constraints, such as healthcare and vehicular networks. Multi-access Edge Computing (MEC) enhances URLLC applications by bringing computation closer to end users, maximizing resource utilization and minimizing latency. This paper introduces Deep Reinforcement Learning for URLLC Optimization in Multi-Edge Networks (DURLLCON), a Deep Reinforcement Learning (DRL)-based framework for efficient task offloading in a hybrid MEC environment, specifically targeting both URLLC and non-URLLC applications. By leveraging DRL and Long Short Term Memory (LSTM) networks, the proposed method dynamically adjusts the offloading decisions based on user energy levels, task priority, and network congestion. The simulation results show that the proposed DURLLCON framework achieves up to 30% improvement in energy efficiency and 20% reduction in average delay compared to existing methods, significantly extending user device battery life while maintaining high Quality of Service (QoS).