Medical entity relation extraction is crucial for building medical knowledge graphs and supporting clinical decision-making. Yet, current models struggle with scarce annotations and evolving domain knowledge. We propose Retrieval-Augmented Relation Extraction (RA-RE), a framework that combines static and dynamic retrieval mechanisms to enhance large language models. The core innovation lies in a Dynamic Semantic Confidence Evaluation (DSCE) module, which adaptively decides whether to retrieve external examples based on prediction reliability. This confidence-aware retrieval enables RA-RE to reduce annotation dependency while improving robustness and accuracy. The dynamic pipeline consists of zero-shot prediction, confidence evaluation, example retrieval, and final prediction, ensuring flexible and knowledge-driven relation extraction. Evaluations on multiple medical datasets and model scales show that RA-RE consistently outperforms strong baselines, highlighting the effectiveness of integrating confidence-aware retrieval into medical information extraction.