A Spike Vision Approach for Multi-object Detection and Generating Dataset Using Multi-core Architecture on Edge Device

Spiking Neural Networks (SNNs) have gained significant attention in the field of neuromorphic computing for their potential to mimic the brain’s spiking neurons, allowing event-driven processing based on exact spike timing. In this paper, we introduce a novel architecture that uses the power of SNN in combination with transfer learning to achieve real-time human presence detection and analysis using event-based cameras and compare it with non-event-based cameras. This architecture, which is deployed on edge computing devices, controls a comprehensive pipeline of components, seamlessly integrating various strategies. It combines object detection, transfer learning with SNN, human recognition, localizing and tracking, feature extraction, multi-core architecture, and run-time analysis. The application is initiated by extensively detecting objects and monitoring environments for motion events. Thus, transfer learning adjusts pre-trained Convolutional Neural Network (CNN) weights to SNNs upon detection, enabling event-driven processing. The utilization of multi-core processing speeds up the analytical workload while maintaining real-time operations. The architecture also keeps a valuable spike train dataset, which records important information about recognized objects. This dataset is useful for applications such as behavioral analysis and real-time monitoring.