Enhancing COVID-19 Literature Retrieval: A Comparative Evaluation of Full Text, Vector, and Hybrid Search Methodologies with Reranking for Domain-Specific Question Answering Systems
摘要
The COVID-19 pandemic has created a perfect storm of scientific literature that has left scholars and healthcare practitioners stranded in sea of data which suffocates lack of clear, operationally useful conclusions. This research explores four search approaches that include Full Text Search (TF-IDF), Vector Search (dense embeddings) and Hybrid Search (BM25 + Vector), and Hybrid Search with Cross-Encoder Reranking, aimed at solving fragmented contexts’ and structure data’s retrieval on COVID question answering. From text fragments from COVID-19 PDFs obtaining 512 tokens each, and questions expert-curated (such as viral life cycles, genetic structures), we evaluated performance on 14 metrics such as Context Precision, Factual Correctness and Semantic Similarity. Results revealed systemic failures: all of the methods reached zero Context Precision/Recall because of broken retrievals and no method reached reasonable BLEU or ROUGE-L values indicating poor alignment with ground truth answers. Factual Correctness saw a marginal increase with the help of Hybrid Search (average: (0.24) but there was no statistical significance, but Vector Search had high Semantic Similarity (peak: 0.45) without factual accuracy. Reranking worsened relevancy scores by more than 99% in cases while prioritising shallow word matching over domain specific context. The work highlights the limitations of the existing methodologies for biomedical QA, endorsing dynamic chunking to maintain cohesion, for the use of entity-level assessment metrics (e.g., SQuAD-style F1 scoring), and domain-specific reranking models. These results can be used as a map for optimizing the retrieval pipelines, which is vital for speeding up scientific discovery and making better healthcare decisions in response to pandemic.