PVEIN: A Pretrained Vertex Embedding Infer Network for Open-Domain Question Answer Scoring
摘要
This study introduces an innovative evaluation methodology designed to enhance the interpretability and efficacy of Open-domain Question Answering (ODQA) systems, which are essential for understanding the intricacies of human language. Unlike traditional metrics that rely on manual annotations and therefore limit evaluation accuracy, our method utilizes a Pretrained Vertex Embedding Infer Network (PVEIN) for a refined analysis of question-answer interactions. By extracting deep semantics through pre-trained models and leveraging graph neural networks to discern complex question-answer relationships, our approach provides a comprehensive analysis. A classification layer, integrated with an explainability module, allows for precise evaluations without extensive manual data. The effectiveness of our methodology is demonstrated through its application to Legal Question Answering and Automated Essay Scoring datasets, where it shows superior performance. The innovation of this work lies in the seamless integration of pre-trained models and graph neural networks, significantly advancing the ODQA evaluation framework in NLP and representing a major step forward in QA system accuracy and interpretability. We will release the code at the following link: https://github.com/WakedupChan/PVEIN .