Large Language Models (LLMs) offer powerful capabilities but their significant size and computational requirements hinder deployment on resource-constrained mobile devices. This paper investigates Post-Training Quantization (PTQ) for compressing LLMs for mobile execution. We specifically apply 4-bit PTQ using the BitsAndBytes library via the Hugging Face Transformers framework to Meta’s Llama 3.2 3B model. The quantized model is further converted to the GGUF format using llama.cpp tools for optimized mobile inference. The proposed PTQ workflow achieved a 68.66% reduction in model size through 4-bit post-training quantization, enabling the Llama 3.2 3B model to run efficiently on a standard Android device. Qualitative validation confirmed the 4-bit quantized model’s ability to perform inference tasks successfully. We demonstrate the feasibility of running the final quantized GGUF model on an Android device using the Termux environment and the Ollama framework. PTQ, particularly down to 4-bit precision combined with mobile-optimized formats like GGUF, presents a viable pathway for deploying capable LLMs directly on mobile devices, balancing model size and functional performance.