Neural Architecture Search (NAS) has a significant impact on the field of computer vision by automatically designing high-performance neural network architectures. However, the substantial computational demand of conventional NAS algorithms (e.g. \(10^4\) GPU hours) makes it difficult to apply them in large-scale tasks. Differentiable Architecture Search (DARTS) is a widely adopted and efficient method for architecture search, enabling the discovery of competitive models within a few days. DARTS and its variants encounter two critical challenges: performance collapse caused by the excessive reliance on parameter-free operations and the discrepancy arising from the conversion of continuous architecture encoding to discrete representation. In this paper, we discuss the root causes of these issues. To address the aforementioned issues in DARTS, we propose a novel loss function, termed VAS loss, designed to mitigate performance collapse and resolve the discrepancy problem. In our experiments, the search requires merely 0.2 GPU-day results in achieving a 2.52 \(\%\) test error on CIFAR-10 and a 24.4 \(\%\) test error on ImageNet.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Variance-Aware Scaling Loss for Enhanced Differentiable Architecture Search

  • Boyu Wang,
  • Yong Zhong

摘要

Neural Architecture Search (NAS) has a significant impact on the field of computer vision by automatically designing high-performance neural network architectures. However, the substantial computational demand of conventional NAS algorithms (e.g. \(10^4\) GPU hours) makes it difficult to apply them in large-scale tasks. Differentiable Architecture Search (DARTS) is a widely adopted and efficient method for architecture search, enabling the discovery of competitive models within a few days. DARTS and its variants encounter two critical challenges: performance collapse caused by the excessive reliance on parameter-free operations and the discrepancy arising from the conversion of continuous architecture encoding to discrete representation. In this paper, we discuss the root causes of these issues. To address the aforementioned issues in DARTS, we propose a novel loss function, termed VAS loss, designed to mitigate performance collapse and resolve the discrepancy problem. In our experiments, the search requires merely 0.2 GPU-day results in achieving a 2.52 \(\%\) test error on CIFAR-10 and a 24.4 \(\%\) test error on ImageNet.