Large language models (LLMs) are advanced neural networks pre-trained on extensive text corpora. A critical aspect of developing LLMs lies in designing effective network architectures that enable the creation of powerful and scalable models. This chapter delves into the architectural foundations of LLMs, with a particular emphasis on the Transformer model, which has become the cornerstone of modern LLMs. We begin by systematically dissecting the Transformer model, examining its components in detail, and explaining how the encoder and decoder modules are constructed and integrated into a unified architecture. Next, we provide a comprehensive overview of the key configurations that define these models, including normalization techniques, positional encoding, activation functions, attention mechanisms, and mixture-of-experts models. Building on these foundational elements, we present a complete configuration of the LLaMA model, accompanied by practical code demonstrations to illustrate its implementation. Furthermore, we explore three prominent architectural paradigms for constructing language models: the encoder–decoder architecture, the causal decoder architecture, and the prefix decoder architecture. To address the challenges of processing long texts, we discuss strategies for extending LLMs to handle longer contexts, including scaling positional embeddings or expanding the context window. Finally, we introduce alternative model architectures based on parameterized state space models, highlighting emerging approaches like Mamba and other variants. This chapter aims to provide a thorough understanding of the architectural principles underlying LLMs, equipping readers with the knowledge to design, implement, and adapt these models for diverse applications.