Conversational Recommender Systems (CRS) aim to provide recommendations through multi-turn natural language interactions. Recent LLM-based approaches have shown impressive performance by leveraging large language models (LLMs). However, these systems face several challenges: they struggle to recommend newly listed items, and are prone to hallucinations. Retrieval-Augmented Generation (RAG) introduces up-to-date knowledge and mitigates hallucinations. Unfortunately, RAG cannot be directly implemented to CRS. First, we found retrieval quality is often hindered by the absence of high-quality query reformulation supervision. To resolve this, we present a pseudo query reformulation method called Compare-then-Summarize (CTS), which eliminates the need for costly manual annotations. CTS-generated queries are used to fine-tune or few-shot prompt an LLM, serving as the query reformulator during inference. Extensive experiments show that CTS supervision is a viable alternative to human annotations, significantly improving retrieval performance. While the retrieval stage reduces hallucination rates, we found some still persist. Moreover, integrating recommendation and response generation remains an open challenge. To tackle this, we introduce Rationalize-then-Recommend (RTR), a fine-tuning task that jointly generates both recommendations and their underlying rationale, ensuring only real items from the valid candidate space are suggested. In Summary, we proposed retrieval-augmented LLM-based CRS that enable the implementation of RAG on CRS, and therefore advancing CRS from LLM-based to RAG-based. Comprehensive experiments confirm that our method consistently outperforms state-of-the-art LLM-based CRS across two widely-used datasets. We open-source our code at https://github.com/dayuyang1999/RAGCRS .