In order to address the challenges of the security measures around railway perimeter, we crafted an efficient deep learning architecture (CBAM-VGG_UNet), which aims to accurately demarcate the railroad perimeter area and effectively extract key information. Deeply integrating the idea of transfer learning, leveraging the pre-trained VGG16 as its encoder and utilizing its extensive features learned on the large dataset of ImageNet, the architecture markedly enhances the segmentation capability of railroad perimeter images. Drawing on this foundation, the CBAM (Convolutional Block Attention Module) module has been ingeniously applied into the VGG_UNet architecture, which enables the model to focus more on the areas that are closely related to the railroad perimeter in the image, thereby further improving the accuracy of the segmentation results. To ascertain the efficacy of the model, the RailVOC dataset was constructed. Subsequent to employing data enhancement techniques on the images, the dataset was utilized for both the training and testing phases of the model. The experimental findings indicate that in terms of MIoU, Recall and MPA, the CBAM-VGG_UNet architecture elicits exceptional performance of 97.88%, 98.96%, and 99.41%, respectively.