The advancement of Artificial Intelligence in recent years has driven the rapid growth of the Industrial Internet of Things (IIoT), bringing a significant increase in complex computing tasks and presenting a substantial challenge to the resource-constrained end devices (EDs) in IIoT. Edge computing and caching strategies have emerged as effective solutions to address this issue by transferring demanding computational tasks and storing frequently accessed data at Edge Access Points (EAPs), ultimately reducing latency and enhancing overall system efficiency. However, due to dynamic environments in IIoT, offloading requests are time varying and stationary offloading and caching algorithms are inefficient. As a result, this paper proposes a multi-agent based jointly offloading and caching (MAJOC) algorithm. Firstly, EAPs in an IIoT are treated as multiple reinforcement learning agents. Then jointly offloading and caching decisions of EDs and EAPs are modeled as Markov decision processes. We formulate the utility function which considers both system costs and cache hit ratio. At last, a MADDPG algorithm is used to maximize the utility function and achieve the optimal offloading and caching scheme in a dynamic IIoT scenario. Results of the simulation demonstrate that the MAJOC algorithm has a remarkable effect on task latency, reducing it by 27.2% and augmenting utility by 22.4%, when compared to benchmark techniques.