With the proliferation of false information on social networks, rumor detection has become a critical and urgent research area. Nonetheless, current rumor detection models mainly rely on textual features, which makes it difficult to fully understand the context and background information. The integration of large language models promises to enhance semantic understanding and provide more comprehensive background information. Unfortunately, existing large language models often fail to provide real background information, thereby limiting their practicality in rumor detection. Moreover, most rumor detection methods focus solely on providing detection labels, neglecting the need for explanations. To address these challenges, we propose a new evidence retrieval rumor detection model called RGED. This model first utilizes a large language model to identify relevant statement entities, then retrieves relevant evidences and summarizes the relationships between a given statement and its evidences. Subsequently, RGED constructs an evidence graph to represent the statements, evidence, and their interconnections. Finally, RGED employs Graph Neural Network, attention mechanisms, and Pre-trained Language Model to detect the veracity of rumors and generates interpretable evidence.