Scholarlink

<p>Neural Architecture Search (NAS) algorithms often incur huge search costs due to the performance evaluation of many candidate networks during the search. To address this challenge, we introduce a novel method in this paper that efficiently explores the architecture search space. Our approach focuses on selectively expanding promising nodes within an architecture tree, leveraging the utilization of two key factors. First, instead of focusing on finding only top-performing networks, we consider multi-objective NAS (MONAS), where conflicting objectives such as the performance and the computational cost of neural architectures need to be considered simultaneously. We also propose a new node-expansion strategy that is beneficial for Multi-Objective Tree-based NAS (MOTNAS). Second, we use low-cost metrics, i.e., the Synaptic Flow and the Jacobian Covariance that require no training epochs, as the performance predictors for quickly identifying potential nodes on the architecture search tree. Our Training-Free Multi-Objective Tree-based NAS (TF-MOTNAS) is then evaluated on the widely-used NAS-Bench-201 benchmark. Experiment results show that TF-MOTNAS outperforms state-of-the-art NAS methods both in terms of the performance of the obtained architectures and the search cost. MONAS yields Pareto sets of multiple networks exhibiting the optimal trade-off between the competing objectives, thereby providing insightful information for decision-makers to choose the appropriate architecture. The code is available at: <a href="https://github.com/ELO-Lab/TF-MOTNAS">https://github.com/ELO-Lab/TF-MOTNAS</a>.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Efficient Multi-Objective Neural Architecture Search via Tree Search with Training-Free Metrics

  • 期刊论文

  • 发表日期：

    2026年2月17日

查看全文
  • An Vo,
  • Nhat Minh Le,
  • Ngoc Hoang Luong

摘要

Neural Architecture Search (NAS) algorithms often incur huge search costs due to the performance evaluation of many candidate networks during the search. To address this challenge, we introduce a novel method in this paper that efficiently explores the architecture search space. Our approach focuses on selectively expanding promising nodes within an architecture tree, leveraging the utilization of two key factors. First, instead of focusing on finding only top-performing networks, we consider multi-objective NAS (MONAS), where conflicting objectives such as the performance and the computational cost of neural architectures need to be considered simultaneously. We also propose a new node-expansion strategy that is beneficial for Multi-Objective Tree-based NAS (MOTNAS). Second, we use low-cost metrics, i.e., the Synaptic Flow and the Jacobian Covariance that require no training epochs, as the performance predictors for quickly identifying potential nodes on the architecture search tree. Our Training-Free Multi-Objective Tree-based NAS (TF-MOTNAS) is then evaluated on the widely-used NAS-Bench-201 benchmark. Experiment results show that TF-MOTNAS outperforms state-of-the-art NAS methods both in terms of the performance of the obtained architectures and the search cost. MONAS yields Pareto sets of multiple networks exhibiting the optimal trade-off between the competing objectives, thereby providing insightful information for decision-makers to choose the appropriate architecture. The code is available at: https://github.com/ELO-Lab/TF-MOTNAS.