Industrial edge computing tasks are delay-sensitive. Service providers often offload tasks to edge servers for processing, while the neglected cloud platform can ensure stable completion in special cases. However, this cloud-edge architecture may lead to a waste of resources and unnecessary latency. In this paper, tasks are guaranteed to be processed in time ensuring moderate utilization of various resources during resource allocation. We consider tightly coupled task scheduling and corresponding heterogeneous resource allocation processes, dividing this process into two subproblems, resource allocation and task scheduling. The asynchronous decision characteristics of the cloud and the edge are considered, so we use an online algorithm to solve these problems. We use the hybrid proximal policy optimization (H-PPO) algorithm for effective resource allocation, which can make a unified decision for various continuous heterogeneous resources. For the task scheduling problem, we used the DQN algorithm. The simulation experimental results show that the proposed method can save resources better and make the delay-sensitive tasks be offloaded in time.