In this study, an optical flow estimation approach using transformer-based deep neural networks is proposed, which consists of spatial feature extraction module, recurrent feature extraction module, matching feature module, coarse matching module, context feature encoding module, context attention module, and recurrent module. Two feature extraction modules extract spatiotemporal features from video frames, matching feature module enhances extracted spatiotemporal features for computing pixelwise correspondences, coarse matching module estimates the initial optical flow, and context feature encoding module and context attention module extract object-level features and propagate motion features to occluded regions, respectively. Finally, recurrent module performs iterative optical flow refinement and generates the final optical flow by upsampling. Based on the experimental results obtained in this study, in terms of three objective performance metrics and subjective evaluation, the performance of the proposed approach is better than those of six comparison approaches.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Optical Flow Estimation Using Transformer-Based Deep Neural Networks

  • Kai-Hsiang Liang,
  • Jin-Jang Leou

摘要

In this study, an optical flow estimation approach using transformer-based deep neural networks is proposed, which consists of spatial feature extraction module, recurrent feature extraction module, matching feature module, coarse matching module, context feature encoding module, context attention module, and recurrent module. Two feature extraction modules extract spatiotemporal features from video frames, matching feature module enhances extracted spatiotemporal features for computing pixelwise correspondences, coarse matching module estimates the initial optical flow, and context feature encoding module and context attention module extract object-level features and propagate motion features to occluded regions, respectively. Finally, recurrent module performs iterative optical flow refinement and generates the final optical flow by upsampling. Based on the experimental results obtained in this study, in terms of three objective performance metrics and subjective evaluation, the performance of the proposed approach is better than those of six comparison approaches.