A social development algorithm for neural architecture search
摘要
The design of deep neural network architectures is crucial in deep learning, with superior architectures yielding exceptional performance. The evolutionary algorithms commonly used in neural architecture search often suffer from local optima and slow convergence. Based on an analysis of human societal development, this paper proposes a social development algorithm (SDA) to address NAS challenges. The novelty of the algorithm primarily lies in maintaining population diversity through the social collaborative operation and elimination and supplementation strategy, which helps avoid local optima. Additionally, the failure tolerance mechanism ensures the search efficiency of the algorithm. Experimental results indicate that the social development algorithm (SDA) improves architecture fitness by 4% within the same timeframe as the Zen-NAS algorithm. We conducted a comparative analysis of SDA against existing state-of-the-art methods on the CIFAR datasets and ImageNet-1K. On the CIFAR-10 dataset, SDA demonstrated superior performance, surpassing the classification accuracy of the Zen-NAS and LEMONADE algorithms by 0.76% and 0.65%, respectively. On the CIFAR-100 dataset, SDA achieved a significant improvement, outperforming Zen-NAS, EffPNet, and AmoebaNet-A by 1.17%, 1.53%, and 1.97%, respectively. On the ImageNet-1K dataset, although EfficientNet-B7 achieved the highest accuracy of 84.4% at a high computational cost of 3800 GPU days, SDA achieved a competitive accuracy of 82.6% with only 0.42 GPU days, demonstrating superior efficiency in neural architecture search. Moreover, SDA outperformed other state-of-the-art methods, showcasing exceptional efficiency and cost-effectiveness in both classification and search capabilities.