<p>Masonry structures are widely used in human building because they are low construction costs and good structural stability. However, cracks in masonry structures pose a significant threat to structural integrity and safety, making accurate detection methods essential. Inefficient visual inspections are not sufficient. Current deep learning approaches, including convolutional neural networks (CNNs) and Transformers, struggle with the complexities of masonry textures, lighting-induced noise and color variations, and the segmentation of small crack regions. Although TransUNet has shown great promise in semantic image segmentation, it has limitations such as ineffective skip connections, loss of detail during up-sampling, and unstable attention mechanisms. To address these issues, we present SO-TransUNet for crack segmentation in masonry structures. The presented model designs a new skip connection method to better combine global context with local detail features. The context broadcasting (CB) mechanism is introduced to mitigate the problem of steep gradient. The up-sampling module is optimized by a dynamic point sampling mechanism. We designed and conducted several experiments on a public benchmark dataset to validate the effectiveness of the presented method. The experimental results demonstrate that our method outperforms TransUNet, U-Net, DeepLabv3+, SETR, and Masonry baselines, achieving superior performance with an mIoU of 82.92% and an F1-score of 81.17%.</p>