Performance Engineering

Fast beats clever. This chapter is a field guide to speeding up PyTorch training and inference with torch.compile, AMP, CUDA Graphs, KV-cache, quantization (torch.ao.quantization, bitsandbytes), and platform-specific tips for CPU/NPU/GPU. The goal: fewer milliseconds, fewer watts, same (or better) accuracy.