Sensor Fusion SLAM with Continual STDP Learning

This chapter proposes a first-of-its-kind SLAM architecture based on both RGB data, and by fusing an event-based camera and a Frequency Modulated Continuous Wave (FMCW) radar for drone navigation. Each sensor is processed by a bio-inspired Spiking Neural Network (SNN) with continual Spike-Timing-Dependent Plasticity (STDP) learning, as observed in the brain. In contrast to most learning-based SLAM systems, the method proposed in this chapter does not require any offline training phase, but rather the SNN continuously learns features from the input data on the fly via STDP. At the same time, the SNN outputs are used as feature descriptors for loop closure detection and map correction. Numerous experiments are conducted to benchmark our system against state-of-the-art RGB methods, and the robustness of the proposed DVS-Radar SLAM approach is demonstrated under strong lighting variations.