Light field (LF) image super-resolution has always been a challenging task due to the complex structure of light field image, where spatial and angular information is highly coupled with varying disparities. Recent studies tend to disentangle a 4D LF image into multiple 2D subspaces, and design domain-specific convolutions to extract LF features from spatial, angular, and epipolar plane image (EPI) subspaces, respectively. However, the limited receptive field range of the convolution operation restricts their ability to learn long-range dependencies of LF features. To this end, this paper proposes cascaded spatial-angular-EPI Transformers for light field image super-resolution (LF-SAET) by integrating the spatial, angular, and EPI Transformers. In addition, to ensure smooth feature transitions between the local and the global feature extraction modules, we introduce an adaptive receptive field (ARF) module. We also adopt a linear attention mechanism to alleviate the significant computational cost when learning spatial information within each sub-aperture image. Extensive experiments on public LF datasets demonstrate the superiority of our method over the state-of-the-art methods.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

LF-SAET: Cascaded Spatial-Angular-EPI Transformers for Light Field Image Super-Resolution

  • Hao Zhang,
  • Junle Yu,
  • Chenyu Wu,
  • Jiahan Meng,
  • Wenhui Zhou

摘要

Light field (LF) image super-resolution has always been a challenging task due to the complex structure of light field image, where spatial and angular information is highly coupled with varying disparities. Recent studies tend to disentangle a 4D LF image into multiple 2D subspaces, and design domain-specific convolutions to extract LF features from spatial, angular, and epipolar plane image (EPI) subspaces, respectively. However, the limited receptive field range of the convolution operation restricts their ability to learn long-range dependencies of LF features. To this end, this paper proposes cascaded spatial-angular-EPI Transformers for light field image super-resolution (LF-SAET) by integrating the spatial, angular, and EPI Transformers. In addition, to ensure smooth feature transitions between the local and the global feature extraction modules, we introduce an adaptive receptive field (ARF) module. We also adopt a linear attention mechanism to alleviate the significant computational cost when learning spatial information within each sub-aperture image. Extensive experiments on public LF datasets demonstrate the superiority of our method over the state-of-the-art methods.