Instance detection as the process of localizing and distinguishing among objects of a given class, such as anatomical structures, has a wide spectrum of applications in medical image analysis. For example, it can be used to address the limitations of semantic segmentation, which struggles with arbitrary image fields of view and overlapping structures. In this study, we investigate the efficiency of detection transformers for three-dimensional (3D) medical imaging, and establish benchmark models for instance detection. Specifically, we employ transformer architectures as backbone networks for multi-scale volumetric feature extraction, and integrate them with our 3D implementation of the detection transformer. Within a cascaded detection-segmentation pipeline, the detected instances are refined through lightweight semantic segmentation networks, resulting in an improved computational efficiency and segmentation accuracy. Our approach leverages self-attention mechanisms for modeling long-range dependencies as well as deformable attention for enhancing salient regions. With the proposed pipeline, we obtained a mean average precision of 0.232 on the publicly available LUNA16 dataset for instance detection of lung nodules, and Dice similarity coefficients of 0.932 and 0.897 on the publicly available Verse2020 and AMOS22 datasets, respectively, for semantic segmentation of vertebrae and abdominal organs. The obtained results outperform current benchmarks, and therefore highlight the potential of transformer architectures for instance detection, particularly when combined with cascaded segmentation strategies.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Transformer-Based Instance Detection in 3D Medical Images

  • Luka Škrlj,
  • Samuel Kadoury,
  • Tomaž Vrtovec

摘要

Instance detection as the process of localizing and distinguishing among objects of a given class, such as anatomical structures, has a wide spectrum of applications in medical image analysis. For example, it can be used to address the limitations of semantic segmentation, which struggles with arbitrary image fields of view and overlapping structures. In this study, we investigate the efficiency of detection transformers for three-dimensional (3D) medical imaging, and establish benchmark models for instance detection. Specifically, we employ transformer architectures as backbone networks for multi-scale volumetric feature extraction, and integrate them with our 3D implementation of the detection transformer. Within a cascaded detection-segmentation pipeline, the detected instances are refined through lightweight semantic segmentation networks, resulting in an improved computational efficiency and segmentation accuracy. Our approach leverages self-attention mechanisms for modeling long-range dependencies as well as deformable attention for enhancing salient regions. With the proposed pipeline, we obtained a mean average precision of 0.232 on the publicly available LUNA16 dataset for instance detection of lung nodules, and Dice similarity coefficients of 0.932 and 0.897 on the publicly available Verse2020 and AMOS22 datasets, respectively, for semantic segmentation of vertebrae and abdominal organs. The obtained results outperform current benchmarks, and therefore highlight the potential of transformer architectures for instance detection, particularly when combined with cascaded segmentation strategies.