PEKD: Joint Prompt-Tuning and Ensemble Knowledge Distillation Framework for Causal Event Detection from Biomedical Literature

Identifying causal precedence relations among chemical interactions in biomedical literature is crucial for comprehending the underlying biological mechanisms. However, several issues persist, including the scarcity of labeled data, the complexity of domain transfer, and limited computing resources in this field. To tackle these challenges, we present a novel approach called Prompt-Ensemble Knowledge Distillation (PEKD). The PEKD model employs a BERT encoder combined with prompt templates to extract causal relationships between events. Additionally, model compression is achieved through a knowledge distillation framework that incorporates loss function regularization constraints, reducing resource overhead and computational time. To enhance the performance of knowledge distillation, an ensemble method with multiple teachers is utilized. Experimental results demonstrate that the proposed approach achieves a significant improvement in macro-F1 compared to the direct distillation methods. Importantly, it exhibits commendable performance when trained on few-shot datasets and compact models.