This paper proposes a novel retrieval method, ReSHQ (Reranking based on the Similarity between Hypothetical query and Query), aimed at improving retrieval accuracy without needing labeled dataset training and reducing reliance on the internal knowledge of language models. ReSHQ generates hypothetical queries from documents using language models and re-ranks retrieval results based on the similarity between the input query and the hypothetical queries. Experiments using the Web search datasets (TREC DL19/20) demonstrated that ReSHQ outperformed the conventional BM25 method, particularly in nDCG@k. While ReSHQ did not surpass HyDE, a conventional method that also does not require labeled dataset training, in overall performance, it achieved comparable or superior accuracy in nDCG@1. Compared to HyDE, ReSHQ exhibited lower retrieval performance with large language models but achieved comparable or superior performance with smaller language models, indicating a reduced reliance on internal knowledge. Experiments using low-resource datasets (BEIR) showed that ReSHQ underperformed compared to BM25. However, the proposed method showed results suggesting its effectiveness in datasets with short document lengths and in question-answering tasks. These results indicate that ReSHQ is effective in Web search datasets and is a promising method, especially for smaller language models. Future challenges include improving the method of generating hypothetical queries and evaluating performance on diverse datasets.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

ReSHQ: Re-ranking Based on the Similarity Between Hypothetical Query and Query

  • Keito Fukuoka,
  • Hisashi Miyamori

摘要

This paper proposes a novel retrieval method, ReSHQ (Reranking based on the Similarity between Hypothetical query and Query), aimed at improving retrieval accuracy without needing labeled dataset training and reducing reliance on the internal knowledge of language models. ReSHQ generates hypothetical queries from documents using language models and re-ranks retrieval results based on the similarity between the input query and the hypothetical queries. Experiments using the Web search datasets (TREC DL19/20) demonstrated that ReSHQ outperformed the conventional BM25 method, particularly in nDCG@k. While ReSHQ did not surpass HyDE, a conventional method that also does not require labeled dataset training, in overall performance, it achieved comparable or superior accuracy in nDCG@1. Compared to HyDE, ReSHQ exhibited lower retrieval performance with large language models but achieved comparable or superior performance with smaller language models, indicating a reduced reliance on internal knowledge. Experiments using low-resource datasets (BEIR) showed that ReSHQ underperformed compared to BM25. However, the proposed method showed results suggesting its effectiveness in datasets with short document lengths and in question-answering tasks. These results indicate that ReSHQ is effective in Web search datasets and is a promising method, especially for smaller language models. Future challenges include improving the method of generating hypothetical queries and evaluating performance on diverse datasets.