Content-aware instruction for zero-shot Chinese criminal legal case retrieval: a large language model-based framework
摘要
As the number of legal cases continues to grow, legal case retrieval (LCR), which aims to identify relevant cases by a given query, plays a more and more important role in legal artificial intelligence (AI). Benefiting from the prosperity of advanced AI techniques, dense retrieval has made great progress in accomplishing the LCR task. However, as a domain-specialized task, building valid LCR systems still faces challenges whereas current studies meet with mixed success. First, the query is generally short and compact, preventing key information from being fully utilized during retrieval. Second, relevance labels are insufficient or even unavailable, resulting in inadequate supervision in model training. Lastly, existing methods have weak generalization ability and are difficult to scale. To tackle these challenges, we propose a novel framework called InstructLCR. We first design an instruction scheme called Content-aware Instruction to guide the instruction-following large language model (LLM) in generating full-content legal cases. Then, query and instruction are combined to form the input of LLM — we add no case example in the input to meet the zero-shot setting. The generated cases provide the retrievers with supplementary context. Aided by the additional contextual information, we can align the queries and candidate cases in a more accurate manner. Moreover, the lack of relevance labels is also alleviated as the generated contents provide more information on which the retrievers can determine the similarity of queries and candidate cases more easily. Experimental results on two Chinese criminal LCR benchmark datasets demonstrate that InstructLCR can boost the retrieval performance by an average of 6.2%, with a maximum improvement of 19%. We have also obtained new state-of-the-art (SOTA) results by simply integrating InstructLCR with existing models, without requiring any training or fine-tuning effort.