Pneumonia, a potentially deadly disease impacting the lungs, is caused by bacterial or viral infections. Early detection is significant in preventing life-threatening outcomes. This paper presents a robust framework for detecting and classifying pneumonia in chest X-ray images by leveraging transfer learning (TL) with convolutional neural networks (CNNs) specifically using MobileNetV2 and DenseNet201 models. The framework is further boosted by integrating the firefly optimization algorithm (FOA) for the feature selection with CNN models. The FOA refines the input features, improving the capability of the CNN models to accurately detect pneumonia. The optimized features are then used by multilayer perception (MLP) and support vector machines (SVMs) classifiers for prediction. Furthermore, we propose an ensemble model (DenseNet201 + MobileNetV2 + SVM) that combines the outputs from all pre-trained models, outperforming individual models. This ensemble model demonstrated exceptional results on the Kaggle chest X-ray pneumonia dataset, achieving an accuracy of 99.35%, precision of 99.48%, sensitivity of 99.22%, specificity of 99.48%, F1-score of 99.35%, and area under the curve (AUC) of 99.65%. Our experimental results determine that our ensemble model outperforms contemporary approaches.