Diagnosing allergies is inherently complex due to its interdependence with varied factors leading to inconsistencies that contribute to unnecessary medical costs, delayed treatments and reduced quality of care. Artificial Intelligence (AI) offers promising approaches to help address these challenges. Ensuring the alignment of data-driven AI diagnoses and human clinical expertise remains challenging. Misaligned AI systems, resulting from small and incomplete data and variability in human classification, can result in undesirable and costly outcomes. To address this, we applied the Comprehensive Abstraction and Classification Tool for Uncovering Structures (CACTUS) to model each of the nine clinicians available, representing their decision-making process. This analysis revealed a strong inter-individual variability in diagnosing allergic diseases and quantified how much each feature contributed to it. Important features tend to have higher variability across the models than not important ones. This ensemble of CACTUS models performed collective classification, which can offer an objective protocol to help clinicians diagnose more consistently.