SpikeCV: open a continuous computer vision era
摘要
SpikeCV is a new open-source computer vision platform designed for the spike camera, a neuromorphic visual sensor that has seen rapid development in recent years. Unlike traditional cameras, the spike camera operates without a global shutter; instead, each pixel accumulates light intensity and asynchronously fires spikes when a preset threshold is reached. These binary spikes can be generated at frequencies up to 40000 Hz, offering a new mode of visual representation with high spatiotemporal resolution, capturing continuous visual information from the environment. Taking advantage of the spike camera’s low latency and high dynamic range, many spike-based algorithms have made significant progress, such as high-quality imaging and ultra-high-speed target detection. To foster a collaborative ecosystem for spike vision, SpikeCV provides ultra-high-speed scene datasets, hardware interfaces, and a comprehensive module library. It emphasizes data encapsulation, standardization of dataset interfaces, modularity for vision tasks, and support for real-time applications in challenging scenarios. Unlike existing vision frameworks, SpikeCV is specifically optimized for spike data and real-time processing, addressing the unique requirements of spike-based systems. With the advent of the open-source Python ecosystem, modules of SpikeCV can be used as a Python library to fulfill most of the numerical analysis needs of researchers. We demonstrate the efficiency of the SpikeCV on offline inference and real-time applications. The project repository address and website are https://openi.pcl.ac.cn/Cordium/SpikeCV and https://spikecv.github.io.