To address the limitations of energy-efficient but computationally limited ARM64-based AI edge devices and general-purpose edge servers, this paper proposes a Decentralized Collaborative Heterogeneous Edge Computing (DCEC) architecture. This architecture combines edge servers with embedded AI devices to achieve low latency and enhanced computational capabilities. Commercially, the DCEC framework is divided into edge private clouds and edge public clouds, aiming to maximize the profits of Edge Service Providers (ESPs) through dynamic resource management. Considering task dynamism, complexity, and resource heterogeneity, this complex problem is formulated as a multi-stage mixed-integer nonlinear programming (MINLP) problem. We developed the ORL-EPM resource management framework—a three-layer optimization system that adjusts task scheduling based on varying latency sensitivity weights and heterogeneous resource demands. Additionally, we introduced a resource collaboration system based on resource leasing to manage resource overloads and accommodate diverse task complexities. This system includes three Economic Payment Models (EPMs) designed to achieve efficient and profitable resource utilization. Extensive simulation results indicate that this method ensures convergence and closely approximates optimal solutions across various scenarios, significantly outperforming existing methods. Testbed experiments demonstrate that the DCEC architecture can reduce latency by up to 21.83% in real-world applications, notably exceeding previous approaches.