We propose a novel approach for video inpainting, which is aimed at the restoration of corrupted video in both spatial and temporal domains. We propose a multi-stage framework that starts by first providing lower-resolution feature representations from corrupted frames using a context encoder. We ensure temporal consistency by invoking a flow completion module, which estimates optical flows between adjacent frames and restores them, considering occlusions caused by masked regions. Bidirectional feature propagation aligns the features of adjacent frames to enhance contextual information for better content synthesis. We employ multi-layer temporal transformers by allowing local and non-local features for filling in missing regions. Finally, we reconstruct the inpainted video sequence at its original resolution via a decoder. The proposed approach is fully differentiable, hence allowing for end-to-end training for high-quality video inpainting. Our approach provides experimental evaluations to demonstrate the quality of restored video.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Restoring Motion: A Video Inpainting Method for Moving Instances

  • Rishabh Shukla,
  • Harkeerat Kaur

摘要

We propose a novel approach for video inpainting, which is aimed at the restoration of corrupted video in both spatial and temporal domains. We propose a multi-stage framework that starts by first providing lower-resolution feature representations from corrupted frames using a context encoder. We ensure temporal consistency by invoking a flow completion module, which estimates optical flows between adjacent frames and restores them, considering occlusions caused by masked regions. Bidirectional feature propagation aligns the features of adjacent frames to enhance contextual information for better content synthesis. We employ multi-layer temporal transformers by allowing local and non-local features for filling in missing regions. Finally, we reconstruct the inpainted video sequence at its original resolution via a decoder. The proposed approach is fully differentiable, hence allowing for end-to-end training for high-quality video inpainting. Our approach provides experimental evaluations to demonstrate the quality of restored video.