LLM Architectures and Foundational Models

With the development environment configured, our focus now shifts to the core component of any LLM system: the model itself. The selection of a base model is a critical decision that dictates the functional capabilities and performance ceiling of the final application. The current landscape of Large Language Models is diverse, with models categorized by several key characteristics. In this chapter, we will conduct a systematic survey of this landscape. We will begin by examining the primary architectural paradigms that grant models their specialized abilities and then analyze the strategic trade-offs between using community-driven open source models and powerful proprietary APIs. The model selection can be broadly put into two phases.