On Contraction Theory Applied to Convolutional Neural Network Regularization

Convolutional neural networks (CNNs) have become increasingly popular in many research fields due to their strong ability to represent complex structures. CNNs are different from other types of artificial neural networks because they incorporate the convolutional element, which improves network performance directly. This unique property has motivated the development of many convolutional models and techniques. Convolutional neural networks are distinguished by their intricate architecture, divided into convolutional layers for feature extraction and fully connected layers for high-level reasoning and classification tasks. This partitioning enables CNNs to effectively capture hierarchical patterns while leveraging learned features for accurate decision-making. However, CNNs still suffer from minor input variations. This study introduces an innovative regularization via Contraction Theory Analysis to enhance convolutional neural network (CNN) stability. In other words, this regularization takes the characteristics that allow to park the contraction in the Fullyconnected part, that is, the output of each layer in FullyConnected converges closely for the inputs belonging to the same class.