Transformer visual tracking via cascaded attention mechanisms

Transformer has shown remarkable advancements in computer vision. The key of Transformer is the self-attention mechanism for capturing long-range dependencies. However, the high similarity in attention maps across heads in Transformer models leads to computational redundancy. To tackle this issue, we design a Cascaded Layer Attention (CLA) to gradually integrate information from different heads. This is achieved by adding the output of the current head to the original input of the next head as a new input. The CLA boosts the model’s capability to capture hierarchical relationships and contextual information, improving its capacity to model long-range dependency relationships. Also, we introduce Token Connection as an adaptive position embedding followed by CLA modules. A self-attention enhancement module (SAEM) is constructed, which includes CLA, token connection, feed forward network and the depthwise convolution. Then, a Feature Aggregation Module (FAM) is designed, which comprises the self-attention enhancement module and cross-attention. Simultaneously, we incorporate an online template updating module for adapting appearance variations. Finally, we propose an end-to-end and one-stream Transformer tracking framework, named CATrack. It achieves superior tracking performance with 52 FPS tracking speed.