Retrieval-Augmented Generation is a powerful framework that enhances language models by retrieving relevant information from large external knowledge bases, relying heavily on high-dimensional vector embeddings. However, storing these embeddings at scale poses significant challenges due to their substantial memory requirements. To address this, we investigate two complementary strategies: quantization of embeddings using low-bit floating point formats, and dimensionality reduction via Principal Component Analysis (PCA). Our experiments, measuring performance using \(nDCG@10\) (the standard MTEB Retrieval benchmark metric for ranking quality) relative to the float32 baseline, show that low-bit formats like float8 achieve 4x storage reduction with minimal (<0.3%) performance loss, significantly outperforming int8 at the same compression level. Notably, combining float8 with moderate PCA (e.g., retaining 50% dimensions) yields even better trade-offs, achieving 8x total compression with less performance degradation than int8 alone (e.g., a 0.62% drop vs 1.53% for nomic-embed-text-v1.5) while requiring only half the storage space. These findings highlight the effectiveness of combining float8 quantization and dimensionality reduction for efficient RAG embedding storage with minimal impact on retrieval quality.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

RAG Embeddings Storage Optimization Through Quantization and Dimensionality Reduction

  • Naamán Huerga-Pérez,
  • Rubén Álvarez,
  • Álvaro Sánchez-Fernández,
  • Javier Díez-González

摘要

Retrieval-Augmented Generation is a powerful framework that enhances language models by retrieving relevant information from large external knowledge bases, relying heavily on high-dimensional vector embeddings. However, storing these embeddings at scale poses significant challenges due to their substantial memory requirements. To address this, we investigate two complementary strategies: quantization of embeddings using low-bit floating point formats, and dimensionality reduction via Principal Component Analysis (PCA). Our experiments, measuring performance using \(nDCG@10\) (the standard MTEB Retrieval benchmark metric for ranking quality) relative to the float32 baseline, show that low-bit formats like float8 achieve 4x storage reduction with minimal (<0.3%) performance loss, significantly outperforming int8 at the same compression level. Notably, combining float8 with moderate PCA (e.g., retaining 50% dimensions) yields even better trade-offs, achieving 8x total compression with less performance degradation than int8 alone (e.g., a 0.62% drop vs 1.53% for nomic-embed-text-v1.5) while requiring only half the storage space. These findings highlight the effectiveness of combining float8 quantization and dimensionality reduction for efficient RAG embedding storage with minimal impact on retrieval quality.