Large language models (LLMs) are pretrained on vast amounts of data, enabling them to amass extensive knowledge within their internal parameters. As knowledge is continually evolving and updating, retrieval-augmented generation (RAG) becomes a popular technique to inject the latest external information into LLMs to generate more accurate responses. However, a notable issue with RAG is the conflict between the external information and LLM’s internal parametric knowledge. Existing solutions to mitigate this issue primarily focus on relying solely on external context or internal knowledge, mitigating conflicts by decoding and amplifying the influence of external context or enhancing the weight of one side of the knowledge, which may lead to ambiguous erroneous answers in scenarios with multiple mutually exclusive sources. To address these limitations, we propose MARAG, which adopts an end-to-end reasoning process that involves ‘knowledge refinement, conflict detection, answer generation’ to effectively resolute the conflicts and generate more accurate responses. MARAG consists of two major collaborative agents: the Focuser Agent and the Option Helper. The Focuser Agent executes relevance selection and refinement of external context and the model’s internal implicit knowledge, while assessing consistency among different information sources to explicitly flag potential conflicting segments as it removes noise from the text. The Option Helper fully utilizes the option structures in the dataset to transform the open-domain question answering (QA) task into semantic consistency matching, while assessing conflict types and generating the final answer based on evidence refined by the Focuser Agent. Through extensive experiments on the mainstream benchmarks, MARAG demonstrates an average accuracy improvement of 2% compared to existing methods across various LLM backbones, significantly reducing the rate of erroneous responses without the need of fine-tuning the LLM parameters.