Topological and Simplicial Features in Reservoir Computing Networks

Reservoir computing is a framework which uses the non-linear internal dynamics of a recurrent neural network to perform complex non-linear transformations of the input. This enables reservoirs to carry out a variety of tasks involving the processing of time-dependent or sequential-based signals. Reservoirs are particularly suited for tasks that require memory or the handling of temporal sequences, common in areas such as speech recognition, time series prediction, and signal processing. Learning is restricted to the output layer and can be thought of as “reading out” or “selecting from” the states of the reservoir. With all but the output weights fixed they do not have the costly and difficult training associated with deep neural networks. However, while the reservoir computing framework shows a lot of promise in terms of efficiency and capability, it can be unreliable. Existing studies show that small changes in hyperparameters can markedly affect the network’s performance. Here we studied the role of network topologies in reservoir computing in the carrying out of three conceptually different tasks: working memory, perceptual decision making, and chaotic time-series prediction. We implemented three different network topologies (ring, lattice, and random) and tested reservoir network performances on the tasks. We then used algebraic topological tools of directed simplicial cliques to study deeper connections between network topology and function, making comparisons across performance and linking with existing reservoir research.