Less Hallucination and More Factuality: Human Values Alignment in Text Summarization

Text summarization task aims to condense lengthy documents into shorter texts while preserving the essential information. However, related work mostly focused on preserving semantic information of documents and ignored the human values alignment between documents and summaries. What’s more, summaries generated by LLMs are often found to contain hallucinations and lack factuality. In this paper, we introduce a two-stage extract-generate summarization framework to address the above issues. In the first stage, sentences containing both semantic and human values are extracted through a diffusion model. Then in the second stage, overall summaries are generated via LLMs. By extracting important sentences before summarization, we can shift LLMs’ attention to preserve more human values information from the documents, hence achieving the goal of human values alignment. Specifically, we follow Schwartz’s Theory of Human Values for human values definition due to its wide adoption as a value framework in the realm of NLP(Natural Language Processing). To verify the performance of our framework, we designed several LLM-only methods including in-context methods as baselines to compare with. Experiments conducted on CNN/DM and Reddit summarization datasets demonstrate improvements on both human values alignment and factuality achieved by our framework, indicating the importance of an extractor before leveraging LLM for summarization.