Bi-atrial segmentation in late gadolinium-enhanced MRI (LGE-MRI) is an important step for diagnosing and treating atrial fibrillation. However, in real scenarios, this process is hugely impeded by varying appearance of the atrial shape, intensity variations and large inter- and intra-class distribution among volumes. In this paper, we propose a 3D-based self-distillation network for robustly segmenting both the left atrium, right atrium as well as their walls. Specifically, our network employs a 3D segmentation model to better explore the spatial dependencies information in input volumes. The model is equipped with an atrous spatial pyramid pooling module to better deal with variation in atrial shape and thin boundary details at different scales followed by a spatial and depth self-attention module to learn the long-range relationship between relevant information within volume. In addition, we apply a distillation training scheme to improve the robustness of the segmentation model for huge intensity variations among volumes. The experimental results show that our framework can produce a robust performance for bi-atrial segmentation and attain an average Dice of 83.3 \(\%\) and HD95 of 4.51 (pixels).