Knowledge Graphs (KGs) face challenges of incompleteness, driving the requirements of Knowledge Graph Completion (KGC). The development of Large Language Models (LLMs) provides a new perspective for KGC research. Several methods instruct LLMs to conduct KGC by Prompt Engineering. However, they struggle with missing entity/relation descriptions, “text mismatch” between LLMs’ responses and entities in KG, and insufficient utilization of structural information in KG. Notably, Retrieval-Augmented Generation (RAG) has emerged as a powerful paradigm, demonstrating promising potential in enhancing LLM performance across diverse tasks by grounding generation in retrieved knowledge. To capitalize on this potential and address the aforementioned KGC-specific limitations, this paper proposes a novel RAG paradigm named Ret-Gen specifically designed for KGC. It retrieves the corresponding subgraph for each query, and then generates additional knowledge to enhance LLMs’ comprehension of query. Based on this paradigm, we propose a muti-stage method named RAG-KGC. It first constructs a subgraph related to query. Then, it generates abundant additional knowledge based on the query and its subgraph to enhance LLMs’ cognition, thereby enabling better inference. Finally, it conducts similarity matching between LLMs’ responses and the entities in knowledge graph to obtain candidate entities, with the aim to solve “text mismatch” problem. We conduct comprehensive experiments on the link prediction task using two benchmark datasets, FB15K-237-N and Wiki27K, which proved the effectiveness of RAG-KGC.