<p>The design of Artificial Neural Networks (ANNs) for classification tasks has been a topic of interest. However, defining an optimal ANN architecture remains challenging, especially when considering resource constraints and the large number of design parameters. This paper proposes an Evolutionary Bi-Level Neural Architecture Search with Training (EB-LNAST) approach that simultaneously optimizes the architecture, weights, and biases of a neural network using a bi-level optimization strategy. The upper level focuses on minimizing the network complexity penalized by the lower level performance function, while the lower level optimizes training parameters to minimize the loss function and maximize the predictive performance. The proposal is evaluated on a real-world color classification task and the WDBC dataset, demonstrating statistically significant improvements over traditional machine learning algorithms, as well as advanced models. Compared to Multilayer Perceptron (MLP) based algorithms, EB-LNAST achieves superior predictive performance when the architecture is fixed, and remains competitive, with a marginal reduction in performance of no more than <InlineEquation ID="IEq1"> <InlineMediaObject> <ImageObject Color="BlackWhite" FileRef="41598_2025_22538_Article_IEq1.gif" Format="GIF" Height="16" Rendition="HTML" Resolution="72" Type="Linedraw" Width="43" /> </InlineMediaObject> <EquationSource Format="TEX">\(0.99\%\)</EquationSource> </InlineEquation>, even when compared against MLPs optimized with extensive hyperparameter tuning, including architecture, activation functions, regularization, and optimizers. Remarkably, EB-LNAST achieves up to a <InlineEquation ID="IEq2"> <InlineMediaObject> <ImageObject Color="BlackWhite" FileRef="41598_2025_22538_Article_IEq2.gif" Format="GIF" Height="16" Rendition="HTML" Resolution="72" Type="Linedraw" Width="51" /> </InlineMediaObject> <EquationSource Format="TEX">\(99.66\%\)</EquationSource> </InlineEquation> reduction in model size, highlighting its ability to discover compact and efficient architectures. EB-LNAST is a reliable alternative for generating compact and effective neural network architectures in accordance with the problem’s requirements, enabling efficient exploration of the search space while maintaining or exceeding the predictive performance of state-of-the-art classification algorithms.</p>

Evolutionary bi-level neural architecture search with training: A framework for color classification

期刊论文
出版模式：

开放获取
发表日期：

2025年11月4日

查看全文

Mitchell Ángel Gómez-Ortega,
Miguel Gabriel Villarreal-Cervantes,
Mario Aldape-Pérez,
Alam Gabriel Rojas-López,
Daniel Molina-Pérez,
Ramón Silva-Ortigoza

摘要

The design of Artificial Neural Networks (ANNs) for classification tasks has been a topic of interest. However, defining an optimal ANN architecture remains challenging, especially when considering resource constraints and the large number of design parameters. This paper proposes an Evolutionary Bi-Level Neural Architecture Search with Training (EB-LNAST) approach that simultaneously optimizes the architecture, weights, and biases of a neural network using a bi-level optimization strategy. The upper level focuses on minimizing the network complexity penalized by the lower level performance function, while the lower level optimizes training parameters to minimize the loss function and maximize the predictive performance. The proposal is evaluated on a real-world color classification task and the WDBC dataset, demonstrating statistically significant improvements over traditional machine learning algorithms, as well as advanced models. Compared to Multilayer Perceptron (MLP) based algorithms, EB-LNAST achieves superior predictive performance when the architecture is fixed, and remains competitive, with a marginal reduction in performance of no more than \(0.99\%\) , even when compared against MLPs optimized with extensive hyperparameter tuning, including architecture, activation functions, regularization, and optimizers. Remarkably, EB-LNAST achieves up to a \(99.66\%\) reduction in model size, highlighting its ability to discover compact and efficient architectures. EB-LNAST is a reliable alternative for generating compact and effective neural network architectures in accordance with the problem’s requirements, enabling efficient exploration of the search space while maintaining or exceeding the predictive performance of state-of-the-art classification algorithms.