Medical text classification is crucial in healthcare but the complexity of medical terminology, the issue of polysemy, confidentiality of patient data and the need for an accurate and robust classification model pose significant challenges. This paper introduces a novel approach designed to overcome these obstacles. We propose an approach that combines advanced text representation techniques, ensemble learning with deep learning models and federated learning to address these problems. Initially, our approach combines advanced language models like GPT and BERT, enabling a thorough analysis of complex medical texts. Then, to further improve the classification performance, we implemented an ensemble learning strategy, leveraging the strengths of multiple deep learning models. Crucially, to ensure data privacy, we incorporated a federated learning framework. This allows multiple hospitals to collaboratively train ensemble learning models without sharing sensitive patient data, thus adhering to data protection regulations. Our comprehensive experiments on a medical text dataset demonstrate that our proposed approach significantly outperforms existing methodologies across key performance metrics, highlighting the effectiveness of our integrated framework in addressing the challenges of medical text classification, particularly in ensuring the security and privacy of patient data.