Electron Microscopy Image Registration with Twin Axial Transformer and Progressive Training

Convolutional Neural Network (CNN) based approaches for electron microscopy (EM) image registration are limited in their effectiveness due to their locality characteristic. Recently, the Vision Transformer (ViT) has significantly advanced computer vision by effectively processing long-distance dependencies in images with its self-attention mechanism. Nevertheless, while ViT is proficient in handling a wide range of images, it encounters difficulties in processing images with large deformations and complex structures in image registration tasks. To address these challenges, in this paper, we present an EM image registration network that leverages the Twin Axial Transformer enhanced with a multi-scale feature extraction module. The integration of multi-level feature extraction with the ViT framework allows for a detailed analysis of visual information at various scales, significantly enhancing the network’s ability to process and align EM images. Furthermore, we introduce a progressive training approach designed to overcome the difficulties associated with training on datasets with large deformations, making the training process more efficient and stable. Lastly, we automatically generate deformed images and their corresponding high-quality optical flow labels to effectively improve the registration accuracy. Experimental results on a private dataset and a publicly available dataset demonstrate that the network proposed in this study significantly improves registration accuracy and efficiency in handling large-scale deformations in EM image registration compared to existing technologies.