Modular Quantitative Temporal Transformer for Biobank-Scale Unified Representations

Transformers provide a novel approach for unifying large-scale biobank data spread across different modalities and omic domains. We introduce Modular Quantitative Temporal Transformer (MQTT), a modular architecture for multimodal data that offers a robust fusion mechanism for biobank data, which can integrate (1) systematically missing modality data, (2) quantitative data, and (3) longitudinal data. We apply the model for the fusion of personal, laboratory, diagnostic, clinical, and drug prescription data from the UK Biobank. We investigate MQTT’s modular representations, focusing on multimorbidity and polypharmacy. We demonstrate its applicability by a novel stratification of major depressive disorder resulting in subtypes related to treatment-resistant depression with novel genes significant at the GWAS level.