A Fast and Scalable Frame-Recurrent Video Super-Resolution Framework

The video super-resolution(VSR) methods based on deep learning have become the mainstream VSR methods and have been widely used in various fields. Although many deep learning-based VSR methods have been proposed, they cannot be applied to real-time VSR tasks due to the vast computation and memory occupation. The lightweight VSR networks have faster inference speeds, but their super-resolution performance could be better. In this paper, we analyze the explicit and implicit motion compensation methods commonly used in VSR networks and design a fast and scalable frame-recurrent VSR network(FFRVSR). FFRVSR incorporates the Frame-Recurrent Network and Recurrent-Residual Network. This network structure can extract information from low-resolution video frames more efficiently and alleviate error accumulation during inference. We also design a super-resolution flow estimation network(SRFnet) that can more accurately estimate optical flow between video frames while reducing error information ingress. Extensive experiments demonstrate that the proposed FFRVSR surpasses state-of-the-art methods in terms of inference speed. FFRVSR also has strong scalability and can be adapted for both real-time video super-resolution tasks and high-quality video super-resolution tasks.