Optimizing SciBERT for Efficient Question Answering Through Quantization
摘要
Scientific Question Answering (QA) systems must reconcile the growing volume and complexity of scientific literature with the need for rapid, accurate information retrieval. While domain-adaptive models such as Scientific Bidirectional Encoder Representations from Transformers (SciBERT) deliver strong performance on scientific QA tasks, their large parameter counts and high-precision representations hinder real-time and edge deployments. In this work, we investigate post-training quantization of SciBERT for scientific QA, aiming to reduce model size and inference latency without substantially degrading answer quality. We apply 8-bit dynamic quantization to both the sentence-level relevance classifier and span-extraction modules of SciBERT. Experiments on multiple QA benchmarks demonstrate a fourfold reduction in memory footprint by 74.83% alongside only a 4–6% drop in accuracy and F1 scores, while achieving significant speedups in CPU inference. We analyze trade-offs between compression and domain-specific performance, providing practical guidelines for deploying efficient scientific QA models in resource-constrained environments. Our findings highlight the viability of quantized SciBERT and suggest future directions, including mixed-precision schemes and hybrid compression strategies.