Scholarlink

The dense retrieval model offers remarkable capabilities, yet it exhibits inconsistencies in the embedding space of queries and documents due to its dual-encoder structure. Addressing this limitation, we introduce Pseudo-query Embedding (PqE), a document expansion approach that eliminates the need for supervised data. By zero-shot prompting large language models (LLMs), we generate a specific number of pseudo-queries for each document, which are used to mitigate inconsistencies in the embeddings between queries and documents. This innovative strategy employs a multi-stage retrieval process to expand documents, enhancing the performance of the dense retrieval model without unduly impacting retrieval time. Experimental results demonstrate the efficacy of PqE. On the TREC DL dataset, PqE enhances the nDCG@10 metric of the unsupervised dense retrieval model Contriever by 7% points, and the Recall@1k metric by 4% points, surpassing the performance of the BM25 algorithm. Even for contriever \(^{\text {FT}}\) , fine-tuned on the massive dataset MS-MARCO, and BGE, trained on hundreds of millions of query-document pairs, PqE boosts their Recall@1k metrics on the TREC DL dataset by 1 to 2% points. Notably, on out-of-domain datasets from BEIR, PqE elevates the performance of most models across all metrics.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

PqE: Zero-Shot Document Expansion for Dense Retrieval with Large Language Models

  • 会议论文

  • 发表日期：

    2025年

查看全文
  • Jiyuan Liu,
  • Dongsheng Zou,
  • Naiquan Chai,
  • Yuming Yang,
  • Hao Wang,
  • Xinyi Song

摘要

The dense retrieval model offers remarkable capabilities, yet it exhibits inconsistencies in the embedding space of queries and documents due to its dual-encoder structure. Addressing this limitation, we introduce Pseudo-query Embedding (PqE), a document expansion approach that eliminates the need for supervised data. By zero-shot prompting large language models (LLMs), we generate a specific number of pseudo-queries for each document, which are used to mitigate inconsistencies in the embeddings between queries and documents. This innovative strategy employs a multi-stage retrieval process to expand documents, enhancing the performance of the dense retrieval model without unduly impacting retrieval time. Experimental results demonstrate the efficacy of PqE. On the TREC DL dataset, PqE enhances the nDCG@10 metric of the unsupervised dense retrieval model Contriever by 7% points, and the Recall@1k metric by 4% points, surpassing the performance of the BM25 algorithm. Even for contriever \(^{\text {FT}}\) , fine-tuned on the massive dataset MS-MARCO, and BGE, trained on hundreds of millions of query-document pairs, PqE boosts their Recall@1k metrics on the TREC DL dataset by 1 to 2% points. Notably, on out-of-domain datasets from BEIR, PqE elevates the performance of most models across all metrics.