MPRL: Multi-Perspective Reinforcement Learning for Enhancing Format Adherence Capability of Large Language Models

The ability to generate structured text like JSON is well-established in large language models (LLMs). However, a pronounced performance gap exists for smaller language models (SLMs) with less than 10B parameters, which often fall short in understanding and adhering to complex schema constraints provided in the input instruction. To systematically study this issue, we construct JSONBench, a benchmark dataset curated through a four-stage cleaning pipeline and covering diverse, complex schema constraints. By analyzing the errors occurring during JSON generation, we find that the majority of failures stem from semantic and structural constraints, such as symbol misuse, nesting errors, and type inconsistencies. Motivated by these observations, we propose a multi-perspective reinforcement learning framework. By modeling JSON schemas to extract intrinsic constraints, we devise a reward mechanism decomposed into syntax, structure, and content. This effectively aligns SLM outputs with strict schema requirements. Experimental results show that our approach significantly improves JSON generation accuracy across multiple model families, including Qwen and Llama, at various scales (3B, 7–8B, 14B). Moreover, unlike the performance degradation in supervised fine-tuning counterparts, our method demonstrates strong generalization on the FOFO benchmark and maintains the capabilities of SLMs on general tasks (GSM8K and MMLU).