<p>The self-attention mechanism plays a pivotal role in the domain of image captioning. However, this mechanism still faces challenges in application: during the encoding stage, it often fails to capture fine-grained pixel-level information in images, which may limit the model’s precise capture of image details; while during the decoding stage, although self-attention facilitates deep semantic understanding and generation, certain abstract semantics or functional words (such as "of") may not have direct corresponding visual regions in images, an aspect that has not been fully considered. In this work, we address the aforementioned issues in the encoder and decoder through two approaches, respectively. During the encoding stage, we propose a Dual-Stage Pixel Attention (DSPA) that eliminates noise by acquiring finer-grained information. During the decoding stage, we introduce Enhanced Visual Context Attention (EVCA) that bridges the gap between text and vision by learning additional global visual information. To validate our model, we conducted extensive experiments on the MSCOCO image captioning dataset and achieved outstanding performance compared to many state-of-the-art methods. Compared to the latest methods, the results on the MSCOCO dataset demonstrate that our Dual-Stage Pixel Transformer (DSPT) achieves competitive performance on both offline and online test sets, with CIDEr score 134.5% on the Karpathy Split and 133.9% on the official Split. Our source code is available at <a href="https://github.com/a2241119724/DSPT">https://github.com/a2241119724/DSPT</a>.</p>