Deep learning (DL) has shown outstanding results in remote sensing data analysis. In this context, as DL models grow in size and complexity, optimizing their computational and memory efficiency has become crucial for scalable deployment and practical use in a wide range of applications, particularly in the processing of high-dimensional data such as hyperspectral imagery (HSI). This chapter presents a comprehensive analysis of three fundamental techniques for reducing model complexity while preserving predictive performance, i.e., quantization, which decreases the numerical precision of model weights and activations; knowledge distillation, which transfers the representational capacity of a large, high-performing teacher model to a smaller student model; and pruning, which removes redundant or less informative parameters to simplify the network architecture. These representative model compression and optimization strategies are widely applicable across different domains, although their relevance is especially pronounced in scenarios demanding efficient inference, such as large-scale model deployment, real-time applications, and resource-aware optimization. Each technique is explored in terms of its theoretical foundations, practical implementation strategies, and the trade-offs between accuracy, latency, and resource utilization in the context of high-dimensional data analysis, particularly in HSI data processing. To evaluate their impact in remote sensing data analysis, the chapter conducts an experimental evaluation of their performance on DL models for HSI data classification.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Efficient Deep Learning for Hyperspectral Image Classification: Quantization, Distillation, and Pruning

  • María B. García-Flores,
  • Carlos Cañada-Rostro,
  • Juan M. Haut

摘要

Deep learning (DL) has shown outstanding results in remote sensing data analysis. In this context, as DL models grow in size and complexity, optimizing their computational and memory efficiency has become crucial for scalable deployment and practical use in a wide range of applications, particularly in the processing of high-dimensional data such as hyperspectral imagery (HSI). This chapter presents a comprehensive analysis of three fundamental techniques for reducing model complexity while preserving predictive performance, i.e., quantization, which decreases the numerical precision of model weights and activations; knowledge distillation, which transfers the representational capacity of a large, high-performing teacher model to a smaller student model; and pruning, which removes redundant or less informative parameters to simplify the network architecture. These representative model compression and optimization strategies are widely applicable across different domains, although their relevance is especially pronounced in scenarios demanding efficient inference, such as large-scale model deployment, real-time applications, and resource-aware optimization. Each technique is explored in terms of its theoretical foundations, practical implementation strategies, and the trade-offs between accuracy, latency, and resource utilization in the context of high-dimensional data analysis, particularly in HSI data processing. To evaluate their impact in remote sensing data analysis, the chapter conducts an experimental evaluation of their performance on DL models for HSI data classification.