Signal-to-Noise Difference as a Correlate of Class Learning in Neural Networks
摘要
There is a debate from recent studies discussing whether neural networks benefit from low or high latent dimensionality. This work instead focuses on the analysis of signal-to-noise processing and highlights a metric for understanding class learning in neural networks. Remarkably, our results emphasize the signal-to-noise difference (SND), rather than the canonical signal-to-noise ratio (SNR), as the key metric for capturing class learning in neural networks. Fisher Linear Discriminant (FLD) presents a theoretical framework to linearly optimize classification according to the SNR. Akin to this procedure, we adapted FLD to linearly optimize SND. By demonstrating a direct connection between optimized SND and model performance, we underline the central role of SND linear discrimination in driving classification accuracy in neural networks. This metric provides a new perspective on neural network interpretability, shedding light on task-specific layer contributions. These contributions deepen our understanding of the mechanisms underlying neural network class learning, paving the way for finding new ways to optimize classification in deep neural networks.