Intelligent support for soil and water conservation compliance: decision assistance capabilities of prompt-enhanced large language models
摘要
Soil and water conservation (SWC) laws and regulations are hierarchically complex and frequently revised, and grassroots compliance urgently needs efficient, intelligent knowledge tools. Locally deployable small language models (SLMs) avoid the computational cost and data-leakage risks of large models, but their capability boundaries in the SWC legal domain and the pathways for enhancing them remain unclear. Using 71 national and local SWC regulations and the newly promulgated Ecological and Environmental Code of the People’s Republic of China as source materials, we built an evaluation dataset spanning four task types—theoretical knowledge, operational specifications, law-enforcement practice, and comprehensive analysis—across three regulatory levels, and evaluated five SLMs (Qwen3-14B, Qwen3-30B, Phi-4, Gemma3-12B, and Gemma4-31B) under local closed-book conditions. Guided by Cognitive Load Theory, we designed a structured prompt (role setting, task identification, reasoning guidance, and output constraints), ablated each component against an equal-length placebo, and compared reasoning guidance with direct provision of statutory text; beyond accuracy and cost, we added process-level metrics including citation hit rate, rule-selection quality, and suspected hallucination. Baseline accuracy ranged from 68.9% to 84.0% and did not rise monotonically with parameter scale. Structured prompts improved accuracy only marginally while doubling hallucination, whereas providing statutory text raised accuracy by 9.2–14.3% and lowered hallucination to near zero. Knowledge provision, not prompt engineering, is therefore the decisive route to reliable answering. Gemma4-31B with statutory text achieved approximately 94% accuracy with near-zero hallucination within three seconds per question, making it the most cost-effective choice for grassroots deployment.