This study introduces an innovative evaluation methodology designed to enhance the interpretability and efficacy of Open-domain Question Answering (ODQA) systems, which are essential for understanding the intricacies of human language. Unlike traditional metrics that rely on manual annotations and therefore limit evaluation accuracy, our method utilizes a Pretrained Vertex Embedding Infer Network (PVEIN) for a refined analysis of question-answer interactions. By extracting deep semantics through pre-trained models and leveraging graph neural networks to discern complex question-answer relationships, our approach provides a comprehensive analysis. A classification layer, integrated with an explainability module, allows for precise evaluations without extensive manual data. The effectiveness of our methodology is demonstrated through its application to Legal Question Answering and Automated Essay Scoring datasets, where it shows superior performance. The innovation of this work lies in the seamless integration of pre-trained models and graph neural networks, significantly advancing the ODQA evaluation framework in NLP and representing a major step forward in QA system accuracy and interpretability. We will release the code at the following link: https://github.com/WakedupChan/PVEIN .

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

PVEIN: A Pretrained Vertex Embedding Infer Network for Open-Domain Question Answer Scoring

  • Kai Chen,
  • Yingping Deng,
  • Qingcai Chen,
  • Dongfeng Li

摘要

This study introduces an innovative evaluation methodology designed to enhance the interpretability and efficacy of Open-domain Question Answering (ODQA) systems, which are essential for understanding the intricacies of human language. Unlike traditional metrics that rely on manual annotations and therefore limit evaluation accuracy, our method utilizes a Pretrained Vertex Embedding Infer Network (PVEIN) for a refined analysis of question-answer interactions. By extracting deep semantics through pre-trained models and leveraging graph neural networks to discern complex question-answer relationships, our approach provides a comprehensive analysis. A classification layer, integrated with an explainability module, allows for precise evaluations without extensive manual data. The effectiveness of our methodology is demonstrated through its application to Legal Question Answering and Automated Essay Scoring datasets, where it shows superior performance. The innovation of this work lies in the seamless integration of pre-trained models and graph neural networks, significantly advancing the ODQA evaluation framework in NLP and representing a major step forward in QA system accuracy and interpretability. We will release the code at the following link: https://github.com/WakedupChan/PVEIN .