Enhancing thyroid disease prediction with improved XGBoost model and bias management techniques

The thyroid gland, a pivotal regulator of essential physiological functions, orchestrates the production and release of thyroid hormones, playing a vital role in metabolism, growth, development, and overall bodily functions. While thyroid nodules are predominantly benign, certain cases may manifest malignancy, leading to thyroid cancer. Despite the generally favorable prognosis, early detection and intervention remain paramount for optimal outcomes. Given the limitations in clinical data, this paper introduces an innovative approach employing a modified XGBoost model that integrates both clinical and molecular properties into a unified predictive model. Utilizing fine-needle aspiration biopsies and gene expression analysis, our model aims to enhance cancer prediction in thyroid nodules. We explore the influence of incorporating high-quality clinical data on the prioritized gene set, employing a nature-inspired method for hyperparameter optimization. The modified XGBoost model emerges as highly recommended for non-parametric feature selection, extraction, and prediction due to its remarkable accuracy. Our proposed model attains an impressive 94.6% accuracy, surpassing other machine learning models such as C5.0 (92.5%), CART (88.7%), CHAID (83.4%), Quest (81.5%), LSVM (77%), and random tree (24.1%). The real-time efficiency of our model is demonstrated in the results section. Experiments showcase the superior performance of combined molecular and clinical data-based prediction models. Emphasis is placed on the importance of scrutinizing the origins of clinical data and maintaining vigilance over variable quality. This study signifies a substantial advancement in thyroid cancer prediction, offering a robust and accurate model that integrates diverse data sources, ultimately contributing to improved patient outcomes.