<p>Large language models have strong general-purpose capabilities, but their high computational and energy costs limit their practicality in many real-world and resource-constrained settings. In contrast, Small Language Models (SLMs) offer a more efficient alternative for domain-specific applications, where targeted expertise and performance are more important. This survey provides a structured overview of methods that enable compact language models to achieve competitive performance without requiring large parameter counts. We first outline the foundational concepts behind modern Transformer-based models, including common pretraining and fine-tuning paradigms. We then review efficiency-oriented techniques such as parameter-level optimization, prompt-based adaptation, and knowledge distillation, highlighting how knowledge and task behavior can be transferred from large models to smaller ones. The survey also discusses recent approaches that augment language models with external memory, retrieval mechanisms, and tool access to enhance reasoning and decision-making without increasing model size. Finally, we briefly outline future research directions, including the role of efficient and augmented models in emerging agent-based and edge-deployed systems. This survey aims to provide researchers and practitioners with a clear methodological roadmap for designing compact and efficient models that are suitable for practical deployment.</p>