The analysis of iconographic scenes on Byzantine seals provides invaluable insights into the empire’s religious, social, and political trends. The segmentation of iconographic elements in Byzantine seals enables the extraction of detailed, quantifiable information about each design element, facilitating digital analysis of iconographic scenes. However, the task is non-trivial due to challenges such as seal deterioration, poor image acquisition, and the lack of data. In this work, we propose a loss function that incorporates structural constraints based on the expected spatial relationships between iconographic elements in the training of a neural network. The results of the proposed method are evaluated quantitatively and qualitatively. They show the benefit of integrating prior knowledge of spatial relationships into the network to improve performance in a challenging semantic segmentation task.