Calibrating the Converted Spiking Reinforcement Learning

Spiking reinforcement learning (SRL) widely receives attention due to its ultra-low power consumption. Since it is hard to train SRL directly, converting Deep Neural Network into Spiking Neural Network (DNN2SNN) has been a commonly used method to train. However, the conversion error exists with deep reinforcement learning (DRL), resulting in performance degradation of SRL. Inspired by the success of calibrated conversion method in classification tasks, we introduce this method into SRL to further improve the performance of the converted SRL. Specifically, we calibrate the number of spikes fired by converted SNN policy through adjusting the initial membrane potential. Experimental results on MuJoCo robot control tasks demonstrate the effectiveness of Conversion + Calibration method in SRL.