Knowledge base question answering (KBQA) aim to answer given question based on facts from a structured knowledge base. Chinese KBQA (CKBQA) limited the task with Chinese. Previous approaches including semantic parsing-based or retrieval-based approaches have limited performance in large scale CKBQA tasks. In this paper, we proposed a Retrieval Augmented Generation (RAG) based CKBQA system including facts retrieval and semantic parsing (SPARQL generation) with large language models (LLM). Our approach aims to maximize the use of the LLM’s information filtering capabilities, thereby reducing the logical demands of SPARQL query generation. Our approach possesses strong scalability, allowing it to adapt to new questions related to newly added facts without requiring additional supervised fine-tuning (SFT) of the LLM. We achieve 1st in CCKS2024 CKBQA competition with F1 scores of 85.15%.