Relation extraction (RE) is a key task in natural language processing (NLP) and a core component of information extraction. It focuses on identifying semantic relations between entities in text. Pretrained language models (PLMs), such as transformer-based models like BERT, XLNet and RoBERTa, have made notable progress in RE. Predictions of relations from these models are provided with varying confidence levels. While high-confidence predictions of relations are generally accurate, low-confidence predictions tend to be less precise and often lead to inaccuracies. The current research question is how to re-evaluate the low confidence predictions to ensure the overall confidence of a PLM. To solve this problem we propose a framework using automatically generated ontology schemas and LLMs. We first propose an algorithm that constructs ontology schemas from the RE datasets (TACRED and ReTACRED). Then we use LLMs to validate these low-confidence predictions through prompting to further improve the precision of final predictions. Experimental results on transformer-based models, GCN and LSTM-based models across two large-scale RE datasets (TACRED and ReTACRED) show significant improvements in precision and overall performance.