<p>The Convolutional Neural Network (CNN) model in underwater acoustic target recognition (UATR) research reveals limitations arising from its inability to capture long-distance dependencies, impeding its capacity to focus on global information within the underwater acoustic signal. In contrast, the Transformer model has progressively emerged as the optimal choice in various studies, owing to its exclusive dependence on the attention mechanism for extracting global features from input data. Limited research utilizing the Transformer model in UATR has relied on an early ViT model, while in this paper, two refined Transformer models, namely Swin Transformer and Biformer, are adopted as the foundational networks, and a novel Swin Biformer model is proposed by harnessing the strengths of the two. Experimental results demonstrate the consistent superiority of the three models over CNN and ViT in UATR, and the Swin Biformer model remarkably attains the highest recognition accuracy of 94.3% evaluated on a dataset constructed from the Deepship database. At the same time, this paper proposes a UATR method based on pre-trained Transformer, the effectiveness of which is underscored by experimental findings as a recognition accuracy of approximately 97% was achieved on a generalized dataset derived from the Shipsear database. Even with limited data samples and more stringent classification requirements, the method maintains a recognition accuracy of over 90%, all while significantly reducing the training duration.</p>