There is a debate from recent studies discussing whether neural networks benefit from low or high latent dimensionality. This work instead focuses on the analysis of signal-to-noise processing and highlights a metric for understanding class learning in neural networks. Remarkably, our results emphasize the signal-to-noise difference (SND), rather than the canonical signal-to-noise ratio (SNR), as the key metric for capturing class learning in neural networks. Fisher Linear Discriminant (FLD) presents a theoretical framework to linearly optimize classification according to the SNR. Akin to this procedure, we adapted FLD to linearly optimize SND. By demonstrating a direct connection between optimized SND and model performance, we underline the central role of SND linear discrimination in driving classification accuracy in neural networks. This metric provides a new perspective on neural network interpretability, shedding light on task-specific layer contributions. These contributions deepen our understanding of the mechanisms underlying neural network class learning, paving the way for finding new ways to optimize classification in deep neural networks.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Signal-to-Noise Difference as a Correlate of Class Learning in Neural Networks

  • Benjamín Pascual-Estrugo,
  • Salva Ardid

摘要

There is a debate from recent studies discussing whether neural networks benefit from low or high latent dimensionality. This work instead focuses on the analysis of signal-to-noise processing and highlights a metric for understanding class learning in neural networks. Remarkably, our results emphasize the signal-to-noise difference (SND), rather than the canonical signal-to-noise ratio (SNR), as the key metric for capturing class learning in neural networks. Fisher Linear Discriminant (FLD) presents a theoretical framework to linearly optimize classification according to the SNR. Akin to this procedure, we adapted FLD to linearly optimize SND. By demonstrating a direct connection between optimized SND and model performance, we underline the central role of SND linear discrimination in driving classification accuracy in neural networks. This metric provides a new perspective on neural network interpretability, shedding light on task-specific layer contributions. These contributions deepen our understanding of the mechanisms underlying neural network class learning, paving the way for finding new ways to optimize classification in deep neural networks.