On-Device and Edge AI

Edge deployments are opinionated: tight memory, bursty compute, restricted runtimes, and privacy requirements that keep data local. This chapter gives you practical export-and-serve paths from PyTorch to phones, browsers, and small accelerators: ExecuTorch for mobile, TorchScript/torch.export for graph capture, ONNX Runtime/TensorRT for high-perf inference, plus WebGPU/WASM notes for the browser. You’ll also get accuracy and size tactics that make models actually fit.