Ensuring precise information retrieval is still a challenge, as traditional keyword-based and vector-based methods struggle to capture semantic relationships for high-quality responses in Retrieval-Augmented Generation techniques. This paper presents Knowledge-Embedded Retrieval Enhancement System, a hybrid Retrieval-Augmented Generation approach integrating knowledge graphs and embeddings with Large Language Models to enhance search accuracy. We propose a structured pipeline comprising triple generation, graph embedding, and an optimized retrieval strategy. Experimental results show that Knowledge-Embedded Retrieval Enhancement System significantly improves precision over traditional vector-based and hybrid Retrieval-Augmented Generation approaches. Specifically, it improves 4.5 times of the accuracy and saves 20% of Large Language Models calling time compared to the state-of-the-arts. Although we sacrifice a bit of computational costs, this approach excels in contextual similarity and semantic comprehension, demonstrating its potential for knowledge-intensive applications.