Ef-kpress: joint event-frame compression and video generation with event cameras for low-bandwidth VR streaming

High Dynamic Range (HDR), high frame rate, and high-resolution video playback enhance the immersive experience in Virtual Reality (VR). However, directly transmitting video frames with these properties in VR streaming is difficult due to substantial bandwidth requirements and space occupation. In this paper, we propose a novel VR streaming framework that employs an event camera guided deep learning method for video generation, leveraging the EF-Kpress system. Within the EF-Kpress system, in conjunction with edge computing, the remote server transmits original video frames and event information to the VR client. Then the final output is obtained through HDR, frame interpolation, and super-resolution models, resulting in a significant reduction in bandwidth usage. To further minimize bandwidth usage, we also propose compression methods for both event information and video frames. Finally, we implement the EF-Kpress framework and conduct evaluations on real datasets. Experimental results demonstrate that the proposed system framework achieves approximately 99.55% bandwidth savings compared to direct transmission of corresponding video frames.