Transformers in PyTorch

Transformers are just tensor plumbing with a pattern: normalize → attend → add/residual → normalize → MLP → add/residual. Once that clicks, the rest – efficient attention kernels, Hugging Face models, long-context tricks, KV caches, and even Mixture-of-Experts – are variations on the theme. This chapter walks from raw PyTorch attention to production-ready transformer usage, then peeks at scaling tactics you’ll want in real systems.