Deep learning classifiers have reached good accuracy often surpassing the conventional classifiers. To provide the robustness needed in real-world applications, classifier fusion has shown potential. Such fusion methods can involve integration at the feature (embedding) level, classifier score/confidence level, or decision level. In this paper, we explore the enhancement of data privacy in ensemble learning through the integration of Fully Homomorphic Encryption (FHE). Recognizing the potential of ensemble methods to boost performance robustly against data variations, we confront the critical challenge of adversarial attacks that could compromise classifier integrity. To this end, we introduce the Privacy-Preserving Quantile Power Transform Classifier (PPQPTC), an innovative algorithm that applies quantile transformation for score distribution adjustment and power transformation to augment linear classification, all within the FHE domain. The PPQPTC algorithm is uniquely designed to securely process data while encrypted, addressing the urgent need for stringent data privacy and security in sensitive applications. We rigorously evaluate the performance of our algorithm across a range of diverse datasets, including healthcare data and the NIST BSSR-1 dataset for biometric fusion. Our findings reveal that the PPQPTC algorithm not only effectively handles imbalanced datasets but also demonstrates the feasibility and adaptability of conducting secure data processing in encrypted domains.