This paper presents a concept and supporting technology for building RAG-based specialized search engines using open-source frameworks and open web data. The Open Web Index (OWI) provides openly accessible web data, while the modular MOSAIC framework is designed to integrate topical OWI partitions obtained to create search applications tailored to specific use cases. MOSAIC-RAG extends this framework with features based on Large Language Models (LLM), such as summarization or re-ranking. Using this infrastructure, special-purpose and domain-specific search applications can easily be developed and experimented with. For demonstration purposes, we present three example applications in the topical domains of science, health, and arts.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Creating Specialized RAG-Based Search Engines Using the Open Web Index

  • Alexander Nussbaumer,
  • Michael Dinzinger,
  • Sebastian Heineking,
  • Gijs Hendriksen,
  • Felix Holz,
  • Saber Zerhoudi,
  • Martin Potthast,
  • Michael Granitzer

摘要

This paper presents a concept and supporting technology for building RAG-based specialized search engines using open-source frameworks and open web data. The Open Web Index (OWI) provides openly accessible web data, while the modular MOSAIC framework is designed to integrate topical OWI partitions obtained to create search applications tailored to specific use cases. MOSAIC-RAG extends this framework with features based on Large Language Models (LLM), such as summarization or re-ranking. Using this infrastructure, special-purpose and domain-specific search applications can easily be developed and experimented with. For demonstration purposes, we present three example applications in the topical domains of science, health, and arts.