Scientific Question Answering (QA) systems must reconcile the growing volume and complexity of scientific literature with the need for rapid, accurate information retrieval. While domain-adaptive models such as Scientific Bidirectional Encoder Representations from Transformers (SciBERT) deliver strong performance on scientific QA tasks, their large parameter counts and high-precision representations hinder real-time and edge deployments. In this work, we investigate post-training quantization of SciBERT for scientific QA, aiming to reduce model size and inference latency without substantially degrading answer quality. We apply 8-bit dynamic quantization to both the sentence-level relevance classifier and span-extraction modules of SciBERT. Experiments on multiple QA benchmarks demonstrate a fourfold reduction in memory footprint by 74.83% alongside only a 4–6% drop in accuracy and F1 scores, while achieving significant speedups in CPU inference. We analyze trade-offs between compression and domain-specific performance, providing practical guidelines for deploying efficient scientific QA models in resource-constrained environments. Our findings highlight the viability of quantized SciBERT and suggest future directions, including mixed-precision schemes and hybrid compression strategies.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Optimizing SciBERT for Efficient Question Answering Through Quantization

  • Tushar Bulla,
  • Pooja Gani,
  • Somashekhar M. Kinagi,
  • Vijaykumar Muttagi,
  • Uday Kulkarni,
  • S. M. Meena

摘要

Scientific Question Answering (QA) systems must reconcile the growing volume and complexity of scientific literature with the need for rapid, accurate information retrieval. While domain-adaptive models such as Scientific Bidirectional Encoder Representations from Transformers (SciBERT) deliver strong performance on scientific QA tasks, their large parameter counts and high-precision representations hinder real-time and edge deployments. In this work, we investigate post-training quantization of SciBERT for scientific QA, aiming to reduce model size and inference latency without substantially degrading answer quality. We apply 8-bit dynamic quantization to both the sentence-level relevance classifier and span-extraction modules of SciBERT. Experiments on multiple QA benchmarks demonstrate a fourfold reduction in memory footprint by 74.83% alongside only a 4–6% drop in accuracy and F1 scores, while achieving significant speedups in CPU inference. We analyze trade-offs between compression and domain-specific performance, providing practical guidelines for deploying efficient scientific QA models in resource-constrained environments. Our findings highlight the viability of quantized SciBERT and suggest future directions, including mixed-precision schemes and hybrid compression strategies.