Scholarlink

<p>As large language models (LLMs) are increasingly used in legal applications, current evaluation benchmarks focus mainly on factual accuracy while neglecting important linguistic aspects such as clarity, coherence, and terminology. To address this gap, we first develop a regression-based framework to evaluate legal text quality, second construct a specialized set of legal questions, and third analyze 49 LLMs using this framework. Our study primarily focuses on Chinese legal texts due to data availability, while the methodology itself remains language-agnostic and adaptable to other domains. We identify three key findings: (1) legal text quality plateaus at relatively small scales, with Qwen2.5 models flattening beyond 7B (72B adds only 2.7%) and Qwen3 models showing an early plateau at 1.7B; (2) engineering choices such as quantization and context length have no statistically significant effect on legal text quality (<InlineEquation ID="IEq1"> <EquationSource Format="TEX">\(p &gt; 0.0167\)</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mi>p</mi> <mo>&gt;</mo> <mn>0.0167</mn> </mrow> </math></EquationSource> </InlineEquation>), supporting cost-efficient deployment; (3) reasoning models consistently outperform base architectures. A significant outcome of our research is the release of a ranking list and trade-off frontier visualization, which highlight the Qwen3 series as the optimal choice for cost–performance trade-offs. This work advances domain-specific evaluation of linguistic quality by integrating multidimensional assessment with data-driven model analysis. We additionally adopt a variance-penalized metric, AdjScore, to robustly assess model performance. Code and models are available at: <a href="https://github.com/lyxx3rd/LegalEval-Q">https://github.com/lyxx3rd/LegalEval-Q</a>.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Legaleval-q: a benchmark for quality evaluation of LLM-generated Chinese legal text

  • 期刊论文

  • 发表日期：

    2026年2月25日

查看全文
  • Yunhan Li,
  • Gengshen Wu

摘要

As large language models (LLMs) are increasingly used in legal applications, current evaluation benchmarks focus mainly on factual accuracy while neglecting important linguistic aspects such as clarity, coherence, and terminology. To address this gap, we first develop a regression-based framework to evaluate legal text quality, second construct a specialized set of legal questions, and third analyze 49 LLMs using this framework. Our study primarily focuses on Chinese legal texts due to data availability, while the methodology itself remains language-agnostic and adaptable to other domains. We identify three key findings: (1) legal text quality plateaus at relatively small scales, with Qwen2.5 models flattening beyond 7B (72B adds only 2.7%) and Qwen3 models showing an early plateau at 1.7B; (2) engineering choices such as quantization and context length have no statistically significant effect on legal text quality ( \(p > 0.0167\) p > 0.0167 ), supporting cost-efficient deployment; (3) reasoning models consistently outperform base architectures. A significant outcome of our research is the release of a ranking list and trade-off frontier visualization, which highlight the Qwen3 series as the optimal choice for cost–performance trade-offs. This work advances domain-specific evaluation of linguistic quality by integrating multidimensional assessment with data-driven model analysis. We additionally adopt a variance-penalized metric, AdjScore, to robustly assess model performance. Code and models are available at: https://github.com/lyxx3rd/LegalEval-Q.