Harnessing the Power of Reinforcement Learning for Language-Model-Based Information Retriever via Query-Document Co-Augmentation

Recent advances have explored the use of language models (LLMs) as retrievers by rewriting either user queries or corpus documents. However, such unidirectional augmentation cannot resolve their lexical and stylistic mismatch, limiting recall and robustness. We propose an LLM-based retriever that jointly augments queries and documents, with both policies optimized via reinforcement learning (RL) and minimal human bias. We show that independently training query and document augmentation offers little gain, even when combined at inference, whereas our bidirectional RL framework enables coordinated learning of both policies. The main challenge lies in jointly updating two interdependent policies with entangled rewards. We address this by introducing a reward sampling strategy and a tailored RL algorithm that supports effective joint optimization. Experiments show substantial improvements in sparse and dense retrieval, especially on challenging domains, along with strong cross-benchmark generalization. Code and extended paper are available on: https://future-item.github.io/CoAugRetriever/ .