Optimizing Passage Retrieval with Dual-Directional Similarity Propagation
摘要
Passage Retrieval (PR) is at the intersection of two research areas: Information Retrieval (IR) and Question Answering (QA). PR employs Natural Language Processing (NLP) techniques to comprehend and process queries expressed in natural language. It then uses IR techniques to search for and select relevant passages from a vast collection. Traditional PR systems often depend on lexical relevance estimation, which is proven less effective than semantic estimation. Irrelevant passages may be mistakenly considered relevant due to keyword overlap despite lacking meaningful correlation with the query. Meanwhile, truly relevant passages might be overlooked if they lack keyword overlap. To address these issues, we propose a novel approach to enhance PR performance. Our method integrates lexical, semantic, and contextual aspects to better understand the precise meaning of passages and queries. We propose a comprehensive framework that integrates advanced concept detection and relevance propagation techniques. By analyzing the semantic relationships between query terms and passage contents, our framework aims to address the limitations of traditional keyword-based methods.