Topological Dynamics of Functional Neural Network Graphs During Reinforcement Learning

This study investigates the topological structures of neural network activation graphs, with a focus on detecting higher-order Betti numbers during reinforcement learning. The paper presents visualisations of the neurotopological dynamics of reinforcement learning agents both during and after training, which are useful for different dynamics analyses which we explore in this work. Two applications are considered: frame-by-frame analysis of agent neurotopology and tracking per-neuron presence in cavity boundaries over training steps. The experimental analysis suggests that higher-order Betti numbers found in a neural network’s functional graph can be associated with learning more complex behaviours.