The rise of large language models has revolutionized the field of natural language processing, demonstrating impressive capabilities across a wide range of tasks. However, the immense size and computational demands of these models often present challenges, particularly in resource-constrained environments such as mobile devices and edge computing applications. In this paper, we provide a comprehensive review of small language models—transformer-based, decoder-only language models with 100 M–5 B parameters—and their unique advantages over their larger counterparts. We explore the architectures, training techniques, and model compression methods that enable small language models to maintain high performance while significantly reducing computational and memory requirements. Furthermore, we discuss the potential applications and use cases for small language models, highlighting their suitability for low-latency inference, privacy-preserving applications, and deployment on edge devices. By examining the current landscape of small language models, this review aims to inform researchers and practitioners on the state-of-the-art in this rapidly evolving field and inspire further advancements in the development of efficient, high-performing language models.