Formal Semantics in XAI: A Categorical Diagrammatic Framework

Machine learning-based AI systems, despite their widespread success, suffer from limited trust and deployment in critical domains due to their black-box nature. In response, this paper introduces a semantic framework of category-theoretic diagrammatic formalism, which focuses on diagrammatization of model components and formal semantic mapping: translating intuition into mathematical derivations. Our analysis proceeds in three stages: (1) structural decomposition, where string diagrams visually represent model components and their interactions; (2) semantic mapping, where semantic interpretations are assigned to each component, particularly, unlike existing DisCoCat frameworks, our work offers a higher-order extension using typed lambda calculus to systematically capture complex semantics like negation and quantification; and (3) compositional reasoning and verification, where categorical morphisms enable traceability and behavioral transparency through diagrammatic transformation rules that ensure formal uniqueness of interpretations. This semantic framework supports interpretable validation of AI models by leveraging string diagrams to intuitively visualize component interactions and compositional information flow.