Constructing a high-performance neural network demands a substantial amount of proficiency in the continuous design and fine-tuning of neural network architectures, which propels the advancement of neural network architecture search (NAS) algorithms. However, considerable evaluation time is still a severe challenge in NAS. In this undertaking, we put forward a novel algorithm based on the level-based learning swarm optimizer (LLSO) for the automatic search of meaningful deep convolutional neural network (CNN) architectures suitable for image classification tasks, named CLCNN. To reduce the evaluation time, a classifier is built to predict the architecture performance, replacing a majority of tedious network training. Considering the characteristics of the classifier, LLSO is adopted to search for superior architecture, in which the particles are classified into different layers according to fitness. Particles in lower layers learn from ones in higher layers to search for global optima. As a result, LLSO not only has fast convergence ability but also has the ability to evolve the population with only the prediction results of the classifier, without knowing the accurate objective values. Experimental findings demonstrate that CLCNN is capable of rapidly discovering satisfactory CNN architectures that achieve competitive quality compared to state-of-the-art approaches.