Unsupervised Event-to-Image Reconstruction Based on Domain Adaptation

Event camera outputs a stream of asynchronous events, which suffering from a lot of noise, sparse texture, and lacking of static background information. Existing event-to-image reconstruction (E2IR) methods mostly adopt supervised learning approaches. However, it’s hard to collect ground-truth images for real events. To tackle this challenge, we present a novel unsupervised E2IR method based on domain adaption (DA) in this paper. First, we design a long-short time event memory integration method to convert the unstructured events into structured tensor. The structured tensor could effectively alleviate the problem of missing background, but is still noisy. The E2IR method requires further learning about how to improve the quality of the structured tensor, similar to what is learned in conventional noisy image enhancement (IE) process. Thus we subsequently propose a novel DA-based strategy by transferring the knowledge from IE to E2IR. To better transfer the knowledge, we additionally design a multi-scale adversarial training method at the image-level features to bridge the domain gap. Experimental results show that the performance of our proposed Multi-scale Domain Adaptation based E2IR (MDAE2I) method is comparable to the state-of-the-art.