With the rapid development of real-time Internet of Things (IoT) systems, the requirement for information timeliness has become increasingly critical. Currently, a large amount of data is generated at edge devices, making it a research hotspot in edge computing to explore how to process this data promptly to ensure the freshness of information at edge devices and improve the user experience of latency-sensitive applications. Offloading data processing tasks generated by edge mobile devices to an edge server for computation has been adopted as a solution to this issue. Edge mobile devices pay a certain cost to purchase computational resources from an edge server. Although the edge server has relatively sufficient computing resources, if too many devices choose to offload their tasks to the server, it can lead to a shortage of edge server resources and prolonged task processing times, thus failing to meet the freshness requirements of mobile devices. Therefore, selecting an offloading and resource allocation strategy that can dynamically adapt to environmental changes is particularly important. In this paper, we propose a SOFT-DDPG algorithm based on deep reinforcement learning, which jointly optimizes task offloading and resource allocation. By modeling the problem as a Markov Decision Process (MDP), the algorithm simultaneously optimizes the two stages of task offloading and resource allocation, ultimately ensuring the freshness requirements of task information on devices. Experimental comparisons with the widely used DDPG algorithm for continuous action spaces, as well as other baseline methods, demonstrate the effectiveness and stability of our proposed approach.