Event Sentence Coreference Identification (ESCI) seeks to organize ambiguous or related event sentences in an article into distinct clusters. This task is particularly challenging because events with similar themes must be separated into different clusters due to differences in time, location and additional details. Our research reveals that both open-source and closed-source large language models fail to effectively discern the subtle differences between event sentences, even when provided with comprehensive annotation guidelines. To address this challenge, we propose a novel framework where a local student model benefits from abstracted separation logic derived from an online black-box LLM. Specifically, we employ progressive reasoning prompts to derive the logic (implicit knowledge) of coreference analysis from a black-box LLM. Then we adapt to the implicit knowledge with a smaller trainable LLM to capture rules of event separation. Finally, we achieve coreference identification with a context-aware greedy clustering method during inference. Extensive experiments are conducted in a multilingual setting, and our method brings %1.98, %2.13, %0.51 extra improvement in English, Portuguese, Spanish than previous best methods, achieving the new state-of-the-art results of the ESCI benchmark.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Supporting Event Sentence Coreference Identification with Progressive Prompt-Guided Implicit Knowledge Distillation

  • Tailai Peng,
  • Rui Chen,
  • Xinran Xie,
  • Dekun Lin,
  • Zhe Cui

摘要

Event Sentence Coreference Identification (ESCI) seeks to organize ambiguous or related event sentences in an article into distinct clusters. This task is particularly challenging because events with similar themes must be separated into different clusters due to differences in time, location and additional details. Our research reveals that both open-source and closed-source large language models fail to effectively discern the subtle differences between event sentences, even when provided with comprehensive annotation guidelines. To address this challenge, we propose a novel framework where a local student model benefits from abstracted separation logic derived from an online black-box LLM. Specifically, we employ progressive reasoning prompts to derive the logic (implicit knowledge) of coreference analysis from a black-box LLM. Then we adapt to the implicit knowledge with a smaller trainable LLM to capture rules of event separation. Finally, we achieve coreference identification with a context-aware greedy clustering method during inference. Extensive experiments are conducted in a multilingual setting, and our method brings %1.98, %2.13, %0.51 extra improvement in English, Portuguese, Spanish than previous best methods, achieving the new state-of-the-art results of the ESCI benchmark.