This study focuses on developing a logical fallacy detector using a fine-tuned large language model (LLM) to classify various logical fallacies in text. We utilize a publicly available logical fallacy dataset (LOGIC) combined with logically sound statements derived from the Stanford Natural Language Inference Corpus in this study. Our methodology involves preprocessing the data, fine-tuning the model, and validating its performance on test data. We demonstrate that an f1-score of 0.79 could be achieved for the 15-class classification task including logically sound statements. Zero-shot chain of thought prompting further improves classification f1-score to 0.81. The logically sound class obtained an f1-score of 0.99, indicating that the method is highly capable of distinguishing non-fallacious statements from fallacious statements. Our results show that the fine-tuned LLM model offers a promising tool for enhancing argument evaluation and promoting critical thinking.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Logical Fallacy Detection in Text: Leveraging Large Language Models for Improving Human Discourse

  • Aravindh Manickavasagam,
  • Danushka Bandara

摘要

This study focuses on developing a logical fallacy detector using a fine-tuned large language model (LLM) to classify various logical fallacies in text. We utilize a publicly available logical fallacy dataset (LOGIC) combined with logically sound statements derived from the Stanford Natural Language Inference Corpus in this study. Our methodology involves preprocessing the data, fine-tuning the model, and validating its performance on test data. We demonstrate that an f1-score of 0.79 could be achieved for the 15-class classification task including logically sound statements. Zero-shot chain of thought prompting further improves classification f1-score to 0.81. The logically sound class obtained an f1-score of 0.99, indicating that the method is highly capable of distinguishing non-fallacious statements from fallacious statements. Our results show that the fine-tuned LLM model offers a promising tool for enhancing argument evaluation and promoting critical thinking.