Semantic segmentation, a key area in computer vision, has extensive applications across various fields. While the increasing demand for ultra-high resolution image segmentation in recent years has posed challenges in accuracy and memory occupation for current segmentation network architectures, a new performance-memory trade-off Attention Based Global-Local Transformer Network (AGLTN) is proposed. The depth of the local branch backbone network is reduced, while the cross-attention mechanism is introduced in the aggregation layer to efficiently fuse the detail information and context information. Moreover, the DelighT transformation is utilized to lighten the model and reduce memory occupation. The experimental results show that our model accuracy is greatly improved after the introduction of the attention mechanism, and the number of parameters of the model is effectively reduced due to the backbone network reduction and the lightweighting operation, which alleviates the memory occupancy and achieves a balance between performance and memory. Finally, we obtain 77.2% mIoU and only need less than 2GB GPU memory.