Streamlining CNNs Architectures Using a New Multi-objective Optimization Model

Recently, convolutional neural networks (CNNs) have seen extensive application across various fields. However, successful CNNs use an immense number of parameters, which lead to some undesirable problems, such as overfitting, the high consumption of a long time, and a significant amount of memory. Indeed, many connections in fully connected layers of CNNs are redundant, meaning they can be removed without significantly compromising accuracy. In this context, our study introduces a novel approach: a multi-objective optimization model that simultaneously minimizes both cross-entropy loss and a related CNN complexity function. This model enables the fine control of each parameter’s contribution within the CNN’s fully connected layers through a set of associated decision variables. To solve the resulting model, we opt for the evolutionary multi-objective algorithm (NSGA-II). Experimental results on two benchmark datasets SVHN and NORB demonstrate the effectiveness of our proposed approach.