In neural network models, hyperparameters have a significant impact on model performance. Currently, the commonly used hyperparameter optimization methods include manual search, grid search, random search, Bayesian optimization, and so on. However, these methods always exhibit some problems such as high computational cost, low convergence rate and poor model performance. Thus, for image classification tasks, a hyperparameter optimization method based on statistical orthogonal design for neural network models is proposed in this paper. With the same number of experiments, the classification accuracy of the proposed method is significantly better than that of grid search, random search, and Bayesian optimization methods for both two-level and three-level orthogonal designs. With the same classification accuracy as grid search, random search, and Bayesian optimization methods, the proposed method has fewer experimental times. Furthermore, the single-factor rotation method and statistical variance analysis technique are also applied to study the effect of different hyperparameters on the performances of the neural network models.