Towards Understanding Graph Neural Networks: Functional-Semantic Activation Mapping

Graph Neural Networks (GNNs) excel at complex, graph-structured tasks by leveraging node features and graph topology through recursive neural message passing. However, their opaque internal mechanisms limit their applicability. We propose the Functional-Semantic Activation Mapping (FSAM) approach, building on the co-activation graph concept but leveraging GNN structure to explore transparency and better understand how deep representations are learned from the input graph structure. FSAM encapsulates network behaviour within a functional semantic graph, representing functional relationships between nodes based on activation values. This graph encodes statistical correlations between neuron activations across GNN layers, revealing neuron relationships from input to output. Our approach involves two experimental setups in node classification tasks: identifying critical neurons for class prediction on each layer and applying graph analysis to study class similarities and their network impact. We also visualise class pair interactions within community layers, enhancing layer-wise understanding. Our findings integrate neural and symbolic representations, demonstrating that FSAM can provide insights into the inner workings of GNNs and improve transparency in the input graph structure.