Pre-trained language models (LMs) deliver strong performance across a wide range of Natural Language Processing (NLP) tasks but remain costly to deploy on embedded devices due to their high memory and compute requirements. A widely used strategy for adapting LMs to resource-constrained devices is aggressive quantization. At low bit-widths, mixed-precision schemes, where different components of the model use different numerical precisions, offer an effective balance between compression and accuracy. In this work, we evaluate the impact of mixed-precision quantization for inference on the BERT language model. Unlike prior studies that often overlook activation quantization, our evaluation systematically explores mixed-precision configurations for both weights and activations. We also examine the effects of quantizing the embedding layer, which is commonly limited to token-weight quantization. Evaluated on the SQuAD and GLUE benchmarks, our approach achieves substantial reductions in memory and computational cost without sacrificing accuracy.