Augmenting Infrequent Relationships in Clinical Language Models with Graph-Encoded Hierarchical Ontologies

Harnessing primary-care data can facilitate earlier clinical interventions via predictive modelling. Nonetheless, the intricacy of medical terminology and the breadth of ontological data often obscure the inner workings of such models. Despite the growing complexity of artificial intelligence methodologies and the pressing demand for medical tools that seamlessly integrate into clinical workflows, this opacity persists. We propose enhancing clinical Bidirectional Encoder Representations from Transformers (BERT) models with graph attention networks that encode diagnosis and medication concept hierarchies derived from primary care data. In 10-fold cross-validation on cardiovascular and respiratory detection tasks, our graph-enhanced model marginally improves F1 performance over baseline BERT. More importantly, our approach surfaces clinically deterministic patterns in patient groups, provides modular visualisations of influential terminal and ancestral medical concepts, and improves clustering of related conditions. Additionally, the hierarchical encoding allows quantitative analysis of edge relevance within and across diagnosis and medical ontologies. Our research shows that injecting structured knowledge graphs into language model architectures can improve performance through domain-specific regularisation. Additionally, the use of class activation maps throughout the approach allows for richer interpretations of predictions by following activation flows along concept relationships. The dual utility of precise ontology encoding and Large Language Models makes our graph-injected clinical language model more accurate and trustworthy, propelling preventive precision medicine forward.