Skin lesion classification has five significant complexities: intra-class variation, inter-class similarity, small regions of interest, and data imbalance. Moreover, photometric anomalies affect them, such as noise and low contrast. Numerous computer-aided skin lesion classification systems using handcrafted features and deep convolutional neural networks (CNNs) exist, but their performance does not meet dermatologists’ expectations. Many techniques are available for increasing the performance of CNNs using attention mechanisms, residual blocks, gated layers, feature fusion of various networks, and ensemble learning. However, these techniques work in the spatial domain. The proposed network works in spatial and frequency domains and uses five pre-trained baseline networks: DenseNet121, VGG16, MobileNet, InceptionV3, and Xception. The features of each convolutional layer in the spatial domain of a baseline network are mapped into DCT coefficients, representing the pyramidal structure of the network. This enables the extracting of features in spatial and frequency domains. Ensemble learning uses the sum of the probabilities for classification. The improvements compared to the second-best of existing methods in terms of AUC, Acc, Spec, Sens, Prec, and F1-scores for ISIC 2018 and 2019 datasets are reported as AUC (1.86%), Acc (2.38%), Spec (0.77%), Sens (2.21%), Prec (6.31%) and F1-score (1.82%), and for ISIC 2019 dataset: AUC (0.38%), Acc (4.49%), Spec (0.22%), Sens(decrease of 1.72%), F1-score (26.05%). Significant improvements of the proposed network over the state-of-the-art networks have been observed on the ISIC 2018 and ISIC 2019 challenge datasets, with substantial gains shown on the ISIC 2019 dataset.