Scholarlink

<p>This paper presents a controlled comparison of prompting strategies for the COLIEE Task 4 legal entailment task under a unified experimental setting. Using a Japanese instruction-following LLM, we compare zero-shot (ZS), zero-shot Chain-of-Thought (ZS-CoT), and structured Chain-of-Thought (Structured-CoT). Across seven test years (2018–2024), the overall (sample-weighted) accuracy improves along the progression ZS <InlineEquation ID="IEq1"> <EquationSource Format="TEX">\(\rightarrow \)</EquationSource> </InlineEquation> ZS-CoT <InlineEquation ID="IEq2"> <EquationSource Format="TEX">\(\rightarrow \)</EquationSource> </InlineEquation> Structured-CoT under the unified setting, suggesting that prompt structure—rather than model changes—drives the gain in this setting. In addition, we report few-shot prompting with similarity-based retrieval and label balancing as a secondary sensitivity analysis over the number of shots. Few-shot effects vary by year and <i>n</i>, and we do not observe a consistent monotonic improvement as <i>n</i> increases.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

KIS: COLIEE 2025 Task 4 Solver Using Japanese LLM

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年3月18日

查看全文
  • Takaaki Onaga,
  • Yoshinobu Kano

摘要

This paper presents a controlled comparison of prompting strategies for the COLIEE Task 4 legal entailment task under a unified experimental setting. Using a Japanese instruction-following LLM, we compare zero-shot (ZS), zero-shot Chain-of-Thought (ZS-CoT), and structured Chain-of-Thought (Structured-CoT). Across seven test years (2018–2024), the overall (sample-weighted) accuracy improves along the progression ZS \(\rightarrow \) ZS-CoT \(\rightarrow \) Structured-CoT under the unified setting, suggesting that prompt structure—rather than model changes—drives the gain in this setting. In addition, we report few-shot prompting with similarity-based retrieval and label balancing as a secondary sensitivity analysis over the number of shots. Few-shot effects vary by year and n, and we do not observe a consistent monotonic improvement as n increases.