In recent years, Vision Transformers have emerged as a prominent model in the field of image classification, owing to their superior ability to model global contextual information and their impressive performance across various tasks. However, Vision Transformers generally suffer from high computational costs, primarily due to the quadratic complexity of the self-attention mechanism, which leads to time-consuming inference and insufficient local feature learning. To address these challenges, this paper proposes a novel Fourier Convolutional Vision Transformer architecture, termed FCFormer. FCFormer enhances representation learning by combining frequency-domain feature extraction with deep integration of multi-level convolution and self-attention. Specifically, the architecture leverages a Fourier frequency-domain processing module to efficiently extract rich frequency features at a low computational cost while highlighting critical regions for representation learning. Additionally, the integrated aggregated perceptual self-attention mechanism effectively reduces computational overhead while preserving the model’s ability to selectively focus on key parts of image data. Furthermore, a locally enhanced gated linear unit is introduced to improve local feature perception and enable multi-dimensional feature aggregation. Experimental results demonstrate that FCFormer achieves a better balance between parameter efficiency and classification accuracy compared to classical CNN and Transformer models in image classification tasks..

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

FCFormer: Fourier Convolution Vision Transformer for Image Classification

  • Jialin Guo,
  • Min Zhi,
  • Yanjun Yin,
  • Qiaozhi Xu

摘要

In recent years, Vision Transformers have emerged as a prominent model in the field of image classification, owing to their superior ability to model global contextual information and their impressive performance across various tasks. However, Vision Transformers generally suffer from high computational costs, primarily due to the quadratic complexity of the self-attention mechanism, which leads to time-consuming inference and insufficient local feature learning. To address these challenges, this paper proposes a novel Fourier Convolutional Vision Transformer architecture, termed FCFormer. FCFormer enhances representation learning by combining frequency-domain feature extraction with deep integration of multi-level convolution and self-attention. Specifically, the architecture leverages a Fourier frequency-domain processing module to efficiently extract rich frequency features at a low computational cost while highlighting critical regions for representation learning. Additionally, the integrated aggregated perceptual self-attention mechanism effectively reduces computational overhead while preserving the model’s ability to selectively focus on key parts of image data. Furthermore, a locally enhanced gated linear unit is introduced to improve local feature perception and enable multi-dimensional feature aggregation. Experimental results demonstrate that FCFormer achieves a better balance between parameter efficiency and classification accuracy compared to classical CNN and Transformer models in image classification tasks..