Medical relation extraction (MRE) is a critical foundation for intelligent healthcare tasks such as clinical text analysis and drug repositioning. Although large language models (LLMs) with massive parameter counts have achieved significant progress in general relation extraction, they still face challenges in the medical domain due to the specialized and fine-grained nature of medical relation types, as well as the scarcity of high-quality annotated data. In contrast, smaller models, while offering the advantage of fine-tunability, are limited by their foundational capabilities, failing to capture certain relatively complex semantic patterns in medical relation extraction. Inspired by human cognitive learning, in which individuals reinforce strengths through self-practice and overcome weaknesses with external guidance and repeated practice, this paper proposes a knowledge distillation (KD) framework named KD4FIRE, which integrates the complementary strengths of larger teacher models and smaller student models. Specifically, KD4FIRE first divides samples into easy and hard categories based on the smaller student model’s performance on an initial seed set. For easy samples, chain-of-thought (CoT) self-reinforcement is applied to consolidate the student model’s existing knowledge. For hard samples, we propose an error-aware chain-of-thought refinement method and an abstraction-based data augmentation approach to explicitly address the student model’s knowledge gaps. Experimental results demonstrate that KD4FIRE significantly improves model performance in low-resource scenarios, providing a practical and efficient solution for fine-grained medical relation extraction.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

KD4FIRE: A Knowledge Distillation Approach for Fine-Grained Medical Relation Extraction in Low-Resource Settings

  • Wenjing Li,
  • Shijie Li,
  • Jintao Tang,
  • Shasha Li,
  • Ting Wang

摘要

Medical relation extraction (MRE) is a critical foundation for intelligent healthcare tasks such as clinical text analysis and drug repositioning. Although large language models (LLMs) with massive parameter counts have achieved significant progress in general relation extraction, they still face challenges in the medical domain due to the specialized and fine-grained nature of medical relation types, as well as the scarcity of high-quality annotated data. In contrast, smaller models, while offering the advantage of fine-tunability, are limited by their foundational capabilities, failing to capture certain relatively complex semantic patterns in medical relation extraction. Inspired by human cognitive learning, in which individuals reinforce strengths through self-practice and overcome weaknesses with external guidance and repeated practice, this paper proposes a knowledge distillation (KD) framework named KD4FIRE, which integrates the complementary strengths of larger teacher models and smaller student models. Specifically, KD4FIRE first divides samples into easy and hard categories based on the smaller student model’s performance on an initial seed set. For easy samples, chain-of-thought (CoT) self-reinforcement is applied to consolidate the student model’s existing knowledge. For hard samples, we propose an error-aware chain-of-thought refinement method and an abstraction-based data augmentation approach to explicitly address the student model’s knowledge gaps. Experimental results demonstrate that KD4FIRE significantly improves model performance in low-resource scenarios, providing a practical and efficient solution for fine-grained medical relation extraction.