Refining ChatGPT for Document-Level Relation Extraction: A Multi-dimensional Prompting Approach

This work explores the efficacy of large language models (LLMs) like ChatGPT and GPT-4 in document-level relation extraction (DocRE). Our work begins with the assessment of the zero-shot capabilities of leading LLMs in DocRE, followed by an in-depth exploration of ChatGPT’s performance through fine-tuning. We introduce Multi-Dimensional-Prompting, a prompting framework inspired by existing symbolic and arithmetic reasoning techniques in LLMs. Our methodology includes: (1) a task decomposition strategy that breaks down DocRE into sequential sub-tasks of entity pair extraction and relation classification; (2) a process decomposition strategy to refine the DocRE logic, enhancing prompts for more efficient processing; and (3) a relation-type decomposition strategy, classifying predefined relation types into categories, each can be processed by specialized models for a comprehensive final outcome. Our methods improve performance on benchmark datasets DocRED and Re-DocRED, with our fine-tuned ChatGPT outperforming current state-of-the-art methods.