As artificial intelligence technology advances rapidly, multi-agent collaborative learning plays a vital role in solving complex tasks. This paper targets key issues in multi-agent collaborative learning, such as information sharing, strategy updating, and balancing exploration and exploitation. We propose a novel collaborative learning algorithm called Deep Collaborative Q-learning (DCQL). By establishing an efficient multi-agent information sharing mechanism, optimizing strategy updating rules, and adopting dynamic exploration strategies, DCQL significantly enhances the overall performance of multi-agent systems and the decision-making ability of individual agents. Experimental results on complex tasks like large-scale model training demonstrate that DCQL outperforms traditional multi-agent reinforcement learning algorithms in cumulative rewards and strategy diversity, offering a new solution for multi-agent collaborative learning in complex application domains such as large-scale model training.