Scholarlink

<p>The propensity of Large Language Models (LLMs) to hallucinate poses a significant challenge in specialized domains with fragmented knowledge, such as ceramic question answering. A critical yet underexplored factor is how the format of retrieved knowledge—structured versus unstructured—affects model accuracy. In this study, we develop a Retrieval-Augmented Generation (RAG) framework for the Chinese ceramic domain. A domain-specific knowledge graph was constructed from heterogeneous sources, and a BERT-based module was employed for hop prediction and path reasoning. The retrieved subgraphs were provided to LLMs either as structured triples or converted textual descriptions. We evaluate four LLMs (GPT-3.5, GPT-4, ChatGLM2-6B, LLaMA-2-7B) on a Chinese ceramic QA benchmark. Across all models, structured triples yield higher accuracy, with ChatGLM2-6B achieving the best performance (97.24%). Smaller models benefit more from structured representations, while larger models process textual descriptions more effectively. These results highlight the importance of knowledge representation for reducing hallucination in domain-specific QA. However, as all experiments are conducted on Chinese datasets within a single domain, the generalizability of our findings across languages and other vertical domains remains an open question. We discuss this limitation and outline directions for future multilingual and cross-domain validation.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

A study on question answering in the ceramics domain using large language models with retrieved triples and generated textual knowledge prompts

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年5月9日

查看全文
  • Qixian Zhang,
  • Fubao He,
  • Kaihua Hu,
  • Juan Li

摘要

The propensity of Large Language Models (LLMs) to hallucinate poses a significant challenge in specialized domains with fragmented knowledge, such as ceramic question answering. A critical yet underexplored factor is how the format of retrieved knowledge—structured versus unstructured—affects model accuracy. In this study, we develop a Retrieval-Augmented Generation (RAG) framework for the Chinese ceramic domain. A domain-specific knowledge graph was constructed from heterogeneous sources, and a BERT-based module was employed for hop prediction and path reasoning. The retrieved subgraphs were provided to LLMs either as structured triples or converted textual descriptions. We evaluate four LLMs (GPT-3.5, GPT-4, ChatGLM2-6B, LLaMA-2-7B) on a Chinese ceramic QA benchmark. Across all models, structured triples yield higher accuracy, with ChatGLM2-6B achieving the best performance (97.24%). Smaller models benefit more from structured representations, while larger models process textual descriptions more effectively. These results highlight the importance of knowledge representation for reducing hallucination in domain-specific QA. However, as all experiments are conducted on Chinese datasets within a single domain, the generalizability of our findings across languages and other vertical domains remains an open question. We discuss this limitation and outline directions for future multilingual and cross-domain validation.