An overview of tensor and matrix decomposition methods, applied to deep neural networks
摘要
Convolutional neural networks (CNNs) are indispensable for feature extraction in various Artificial Intelligence tasks. However, their computational complexity and memory requirements pose significant challenges. To address these issues, there is a growing need for CNN simplification techniques. One promising approach involves compressing the weight matrices of fully connected layers and the tensors of convolutional layers using several decompositions into matrix products. To this end we investigate the effectiveness of several relevant techniques and their impact on the achieved accuracy, applying the techniques on already trained networks, or during their training, when applicable.