Exploring Emergent Properties of Recurrent Neural Networks Using a Novel Energy Function Formalism

The stability analysis of dynamical neural network systems typically involves finding a suitable Lyapunov function, as demonstrated in Hopfield’s famous paper on content-addressable memory networks. Another approach is to identify conditions that prevent divergent solutions. In this study, we focus on biological recurrent neural networks (bRNNs), specifically the Cohen-Grossberg networks that require transient external inputs. We propose a general method for constructing Lyapunov functions for recurrent neural networks using physically meaningful energy functions. This approach allows us to investigate the emergent properties of the recurrent network, such as the parameter configuration required for winner-take-all competition in a leaky accumulator design, which extends beyond the scope of standard stability analysis. Furthermore, our method aligns well with standard stability analysis (ordinary differential equation approach), as it encompasses the general stability constraints derived from the energy function formulation. We demonstrate that the Cohen-Grossberg Lyapunov function can be naturally derived from the energy function formalism. Importantly, this construction proves to be a valuable tool for predicting the behavior of actual biological networks in certain cases.