<p>Large Language Models (LLMs) have demonstrated impressive capabilities on general language tasks but remain limited in complex reasoning. Current methods often employ multi-agent systems, with LLMs functioning as the core, to enhance reasoning via incorporating more roles and interactions. However, these methods neglect the possible accumulation and transfer of hallucinations during the interaction process. To solve this problem, we propose an Experience-enhanced Multi-Role Debate method based on a Compromise Mechanism (EMRDCM). Inspired by the idea of ensemble learning, we design a diverse multi-agent system by differentiating the roles to improve the collective decision-making effect. An empirical system is also introduced to collect historical data serving as a few-shot prompt to strengthen individual role performance. The proposed compromise mechanism allows the roles to function flexibly, reducing the likelihood of generating and transmitting hallucinations within the agents and ultimately improving reasoning capabilities. Compared to current methods, our compromise mechanism introduces a structured, role-based framework where each agent has specific functionality. This clear role differentiation ensures that the output of one agent is evaluated and validated by another agent before transmission. The introduction of this validation method can detect and correct errors at every stage of the interaction process, significantly reducing the risk of illusion propagation. Our experiments on standard datasets demonstrate the effectiveness of EMRDCM. The results show that EMRDCM outperforms existing state-of-the-art methods in both reasoning performance and anti-hallucination ability.</p>