<p>Edge and cloud collaborative computing is a burgeoning technology to provide computing power support and power Artificial Intelligence (AI) applications. The edge-cloud collaborative systems face the challenge that different control actions work at different timescales, hindering fine-grained energy management. This paper proposes a dual-agent online control framework to strike the right balance between system energy consumption and AI task processing delay. Specifically, Deep Q Network (DQN) agents operate on a large-timescale to determine edge server computation configuration, while Deep Deterministic Policy Gradient (DDPG) agents work on a small-timescale to conduct AI task offloading. The two types of agents are coordinated through cross-timescale state coupling and reward aggregation, enabling joint optimization of task offloading and computation configuration. Moreover, this paper introduces a queue-overflow-aware training mechanism (QO) to penalize severely congested trajectories during centralized training, thereby improving convergence efficiency and enhancing the congestion awareness of the learned policy under finite queue-capacity constraints. Extensive simulations show the proposed approach is able to reduce the average system energy and task completion time by up to 26.1% and 71.3%, respectively, as compared to the dual-agent without QO and single-agent approaches. The approach substantially accelerates convergence in the case of 60–120 gap between small- and large-timescale agents.</p>