The rapid growth of Industrial Internet of Things (IIoT) applications presents significant challenges for distributed edge computing systems, particularly in managing unpredictable and uneven task arrivals that lead to increased latency and reduced system performance. To address this problem, we propose a decentralized Multi-Agent Reinforcement Learning (MARL) framework for adaptive load balancing in heterogeneous edge environments. In this framework, each edge server is modeled as an autonomous agent engaged in a stochastic game, enabling distributed task scheduling and local resource allocation without centralized coordination. The system accounts for realistic factors such as Poisson-distributed task arrivals and heterogeneous computational capacities. Using a Q-learning-based MARL strategy, agents learn to minimize long-term load imbalance through local observations, constrained action spaces, and decaying exploration mechanisms. Simulation results show that the proposed method significantly reduces task latency and load imbalance compared to traditional scheduling baselines, demonstrating its effectiveness in dynamic, large-scale IIoT scenarios.