PTEKC: pre-training with event knowledge of ConceptNet for cross-lingual event causality identification

Event causality identification (ECI) aims to identify causal relations between events in texts. Although existing event causality identification works based on fine-tuning pre-trained language models (PLMs) have achieved promising results, they suffer from prohibitive computation costs, catastrophic forgetting of distributional knowledge, as well as poor interpretability. Particularly in low-resource and cross-linguistic scenarios, existing multi-lingual models are generally confronted with the so-called curse of multilinguality, language bias, and hence result in low accuracy and generalization ability. In this paper, we propose a paradigm, termed Pre-training with Event Knowledge of ConceptNet (PTEKC), to couple Multi-lingual Pre-trained Language Models (mPLMs) with event knowledge for cross-lingual event causality identification. Specifically, we have develop a parameter-sharing adapter plugin that facilitates the integration of event knowledge into the frozen PLMs. This approach significantly diminishes the number of trainable parameters and greatly reduces the risk of catastrophic forgetting. Our Adapter integrates multi-lingual alignment event knowledge into the mPLMs through two designed pre-training tasks, namely event masking and self-supervised link prediction. Extensive experiments on the benchmark dataset MECI show that PTEKC is parameter-efficient and can effectively incorporate multi-lingual alignment event knowledge for improving cross-lingual event causality identification.