Event-Based Object Recognition Using Feature Fusion and Spiking Neural Networks

Event-based cameras have garnered growing interest in computer vision due to the advantages of sparsed spatio-temporal representation. Spiking neural networks (SNNs), as representative brain-inspired computing models, are inherently suitable for event-driven processing. However, event-based SNNs still have shortcomings in using multiple feature extraction methods, such as the loss of feature information. In this work, we propose an event-based hierarchical model using feature fusion and SNNs for object recognition. In the proposed model, input event stream is adaptively sliced into segment stream for the subsequent feature extraction and SNNs with Tempotron rule. And the model utilizes feature mapping to realize the fusion of the orientation features extracted by Gabor filter and spatio-temporal correlation features extracted by the clustering algorithm considering the surrounding past events within the time window. The experiments conducted on several event-based datasets (i.e., N-MNIST, MNIST-DVS, DVS128Gesture and DailyAction-DVS) show superior performance of the proposed model and the ablation study demonstrates the effectiveness of feature fusion for object recognition.