Pneumonia cases pose a significant health challenge due to their infectious nature and associated morbidity. The manual examination of X-ray images is both time-consuming and reliant on the expertise of radiologists. In this paper, we introduce an automated system designed to aid radiologists in distinguishing between Pneumonia, COVID-19, and normal cases using Chest X-Ray images. In a three-staged pipeline, we proposed a trainable feature selector with correlation-guided feature selection, from the features extracted in the first stage. The selected features then aids the classification task at the final stage classifier. The proposed framework achieved an accuracy of 97.05%. We also evaluated the accuracy of our proposed framework by using various state of the art deep learning models in the feature extractor stage and different machine learning model in the classifier stage. We found ResNet-18 as feature extractor in combination with SVM as the classifier gives the best accuracy in our framework.