In this chapter, we address long-term time-averaged performance optimization in dynamic wireless networks, which is usually formulated as a stochastic optimization problem with high computational complexity. We introduce a Lyapunov-guided DRL framework that leverages the Lyapunov optimization method to decompose the multi-stage stochastic problem into a sequence of per-slot subproblems. Each subproblem is then solved adaptively using DRL based on real-time network state observations. The case study focuses on a UAV-assisted wireless network, where GUs report sensing data to a remote BS via UAVs’ relaying. Both GUs and UAVs are equipped with semantic extraction capabilities, enabling them to distill meaningful information from raw sensing data and thus improve transmission efficiency. We formulate a stochastic semantic-aware AoI minimization problem by jointly optimizing UAV-GU associations, semantic information extraction, and UAVs’ trajectories under mobility and communication constraints.