CancersQA: Federated Learning with Pre-trained Models for Intelligent Medical Diagnosis

Pre-trained Models (PTM) have demonstrated extraordinary efficacy in the domain of natural language processing (NLP), particularly in addressing critical challenges within question-answering (QA) systems. However, the medical QA field grapples with issues such as the low robustness of generation models and a shortage of available medical data, which is further complicated by stringent privacy requirements. Leveraging the synergy between Federated Learning (FL) and PTM, we propose an innovative CancersQA method for intelligent cancer diagnosis and effective aggregating algorithm WeiPro. This method employs the GPT-2 model in an FL framework and fine-tunes it using sparse and unevenly distributed medical consultation data. Our novel method is designed to perform optimally on small sample datasets, markedly reducing training time. Our WeiPro aggregation strategy consolidates the parameters of the FL clients by taking into account various factors. Finally, we conducted rigorous experiments to evaluate CancersQA’s performance. Our experimental results indicate that our approach significantly surpasses other models in medical QA tasks underscoring its superior potential for adapting large language models to healthcare’s critical domain.