Model Training and Inference Optimization

Model training and inference optimization is crucial for environmental sustainability in the rapidly expanding field of generative AI. As AI models grow in complexity and scale, their environmental impact becomes increasingly significant, with training and inference processes consuming substantial computational resources and energy. This chapter dives into a few ways to manage and optimize your models for deployment and inference. Specifically, the focus is on neural networks—the most difficult architecture to manage due to its abundant parameters and memory required, as illustrated in Figure 6-1.