Scholarlink

With the explosion of use of Generative AI, deploying large language models (LLMs) in private environments and making them understand proprietary data has become a necessity. Retrieval augmented generation (RAG) in combination with vector databases have become the leading technique for enhancing LLMs with enterprise data. In this paper, we evaluate different vector databases (Chroma, Qdrant, FAISS, and Pinecone) combined with LLM models, such as OpenAI’s GPT-4o mini, in answering multiple questions based on multiple documents. Specifically, we process the data privacy policies of over 100 EdTech providers and use an LLM to answer 45 questions related to GDPR with a yes/no. By evaluating the speed of the vector databases, their technical requirements, and the consistency of their responses, this study provides practical guidelines in selecting the vector database for several use-cases. The results show that Chroma and Qdrant are significantly faster than Pinecone and FAISS, while being consistent in the selection of text chunks based on which questions are answered.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Evaluation of Vector Databases and LLMs in RAG-Based Multi-document Question Answering

  • 会议论文

  • 发表日期：

    2025年

查看全文
  • Elena Filipovska,
  • Ana Mladenovska,
  • Jovana Dobreva,
  • Dimitar Kitanovski,
  • Goran Mitrov,
  • Petre Lameski,
  • Eftim Zdravevski

摘要

With the explosion of use of Generative AI, deploying large language models (LLMs) in private environments and making them understand proprietary data has become a necessity. Retrieval augmented generation (RAG) in combination with vector databases have become the leading technique for enhancing LLMs with enterprise data. In this paper, we evaluate different vector databases (Chroma, Qdrant, FAISS, and Pinecone) combined with LLM models, such as OpenAI’s GPT-4o mini, in answering multiple questions based on multiple documents. Specifically, we process the data privacy policies of over 100 EdTech providers and use an LLM to answer 45 questions related to GDPR with a yes/no. By evaluating the speed of the vector databases, their technical requirements, and the consistency of their responses, this study provides practical guidelines in selecting the vector database for several use-cases. The results show that Chroma and Qdrant are significantly faster than Pinecone and FAISS, while being consistent in the selection of text chunks based on which questions are answered.