<p>Mobile edge computing meets stringent latency requirements by offloading computational tasks to edge servers. However, in dynamic and uncertain environments, assigning tasks to multiple users becomes complex. To address this challenge, we design a multi-user task offloading framework that allows users to initiate service requests in a distributed manner. Specifically, we propose an online learning offloading algorithm based on a distributed auction multi-armed bandit, which can adapt to stochastically changing environments and gradually reduce computational latency. We then transform the dynamic task allocation problem into an online multi-user multi-armed bandit problem and develop an offloading algorithm based on heterogeneous distributed multi-armed bandit (HD-MAB) to optimize user rewards subject to network latency. We demonstrate that the HD-MAB algorithm can achieve optimal task allocation, thereby providing near-optimal service performance with linear regret. Simulation results show that our offloading method performs well in optimizing latency-sensitive tasks, and that user participation in the decision-making process of the HD-MAB algorithm does not affect the asymptotic optimality of the algorithm.</p>