Deep Neural Networks (DNNs) are very successful learning methods when extremely large dataset is available as traditional machine learning (ML) methods are not adaptive to increasingly growing digitization of data and therefore cannot take full advantage of it in improving performance. We will focus on two main issues of most popular DNN that is Convolution Neural Network (CNN) in this paper. The first one is to automate the selection of optimal architecture and hyperparameters of CNN simultaneously using Particle Swarm Optimization method. It will reduce human efforts for finding optimal configuration of CNN and improves performance. Secondly, as optimization of CNN using PSO is a computationally expensive method therefore we used transfer learning (TL) to explore to generalize the ability of PSO-optimized-CNN in solving new problems. The results obtained on three standard datasets of MNIST, CIFAR-10 and CIFAR-100 is encouraging and satisfies the objectives of proposed method.