Accurate segmentation of atrial anatomical structures from late gadolinium-enhanced magnetic resonance imaging (LGE-MRI) has emerged as a vital tool for mapping fibrosis and guiding targeted therapies, thus improving patient outcomes. In this paper, we investigate the application of CNN-based and transformer-based architectures in the MICCAI 2024 MBAS Challenge, building upon the self-configuration and implementation of nnUNet. The five-fold cross-validation results on the training data indicate that CNN-based segmentation models consistently outperform transformer-based models. In line with the conclusion, the official evaluations show that CNN-based models deliver better performance compared with incorporating transformer-based models, with an average Dice score of 0.6640, 0.8806, and 0.9099, and an average HD95 of 4.5258, 7.6845, and 4.9136 for the atrium wall, left atrium cavity, and right atrium cavity, respectively.