The reasoning ability on Table-based Fact Verification (TFV) has been explored extensively on Large Language Models (LLMs). Existing TFV approaches can be broadly categorized into two main paradigms: prompting and fine-tuning. However, most prompting methods highly depend on powerful closed-sourced LLMs, which causes data breach problems, while fine-tuning methods mainly focus on generating end-to-end answers but often lack explainability. In this paper, we introduce a three-stage pipeline framework LEAP for TFV. It provides a step-by-step solution through instruction tuning by decomposing the statement into sub-questions. It consists of three sub-modules: a sentence decomposer, a table-based question answerer, and a table-based evidence-augmented verifier. To specialize the sub-modules in their respective tasks, we construct the reasoning evidence by data distillation and pseudo-labeling. Through diverse experiments on four TFV benchmarks, we demonstrate that our LEAP achieves state-of-the-art performance. In particular, it even outperforms human performance with an accuracy of 2.02% on Infotabs. As an effective method suitable for scenarios that require data privacy, our LEAP framework exhibits strong generalization and versatility on different types of backbone models and datasets. Furthermore, our experiments validate the effectiveness of each sub-module, proving its suitability as a standalone model for its respective task.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

LEAP: An LLM-Based Evidence Augmented Pipeline for Table-Based Fact Verification

  • Hanwen Zhang,
  • Qingyi Si,
  • Peng Fu,
  • Zheng Lin,
  • Zhigang Lu,
  • Weiping Wang

摘要

The reasoning ability on Table-based Fact Verification (TFV) has been explored extensively on Large Language Models (LLMs). Existing TFV approaches can be broadly categorized into two main paradigms: prompting and fine-tuning. However, most prompting methods highly depend on powerful closed-sourced LLMs, which causes data breach problems, while fine-tuning methods mainly focus on generating end-to-end answers but often lack explainability. In this paper, we introduce a three-stage pipeline framework LEAP for TFV. It provides a step-by-step solution through instruction tuning by decomposing the statement into sub-questions. It consists of three sub-modules: a sentence decomposer, a table-based question answerer, and a table-based evidence-augmented verifier. To specialize the sub-modules in their respective tasks, we construct the reasoning evidence by data distillation and pseudo-labeling. Through diverse experiments on four TFV benchmarks, we demonstrate that our LEAP achieves state-of-the-art performance. In particular, it even outperforms human performance with an accuracy of 2.02% on Infotabs. As an effective method suitable for scenarios that require data privacy, our LEAP framework exhibits strong generalization and versatility on different types of backbone models and datasets. Furthermore, our experiments validate the effectiveness of each sub-module, proving its suitability as a standalone model for its respective task.