Learning in Recurrent Spiking Neural Networks with Sparse Full-FORCE Training

Recurrent Spiking Neural Networks (RSNNs) are bio-plausible computational models to detect temporal patterns in data and mimic nonlinear dynamical systems. Due to their temporal information processing structure, neurons in RSNNs need to process a significantly large number of spikes, which increases the energy consumption of inference performed on a neuromorphic hardware. We propose FFSR, a full-FORCE sparsity regularization based supervised training algorithm for RSNNs. The key idea of FFSR is to use a sparsity regularization (SR) technique involving sparse vector optimization to regulate the firing rate of each neuron using a threshold. Supervised learning is facilitated using full-FORCE, which uses a recursive least square-based First-Order and Reduced Control Error (FORCE) algorithm to train a full RSNN structure involving both reservoir and readout. FFSR suppresses information redundancy using SR while improving performance and training convergence using full-FORCE. We evaluate FFSR using seven workloads. Our results show an average 30.5% reduction in the mean square error and an average 53% reduction in the training convergence time with respect to the baseline full-FORCE algorithm.