Ancient murals are invaluable cultural heritages. Due to the influence of natural and human factors, these precious murals are suffering from varying degrees of damages. Existing mural damage detection approaches still struggle to detect multi-scale, blurry, or irregular damage regions. In this paper, we propose a Mask-Guided Multi-Scale Edge-Enhanced Segment Anything Model (MMESAM) for ancient mural damage detection. We uses Segment Anything Model (SAM) as the backbone, with SAM-ViT-B/16 as the pre-trained block. We aims to improve the generalization capability and detection accuracy of the proposed model. Firstly, we design a Feature-Guided Multi-scale Adapter (FGMSA) that allows SAM to extract multiscale information and meanwhile use minimal parameters. Then we introduce a Multi-Stage Feature Fusion Module (MSFM) that can integrate features across encoder layers. Next, we utilize an Edge Enhancement Module (EEM) to incorporate SAM with foreground features and edge details. We conduct the experiments on two self-built datasets of the Mogao Grottoes murals of Dunhuang and the ethnic minority murals of Yunnan. Experimental results show that our model can accurately detect the damaged regions in ancient murals, and outperforms existing approaches in both subjective visual quality and objective evaluation metrics. The EM scores of our model on the two datasets can reach 0.927 and 0.756, respectively, both significantly higher than other comparison approaches.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Mask-Guided Multi-scale Edge-Enhanced Segment Anything Model for Ancient Mural Damage Detection

  • Keming Zhou,
  • Ying Yu,
  • Jialv Hu

摘要

Ancient murals are invaluable cultural heritages. Due to the influence of natural and human factors, these precious murals are suffering from varying degrees of damages. Existing mural damage detection approaches still struggle to detect multi-scale, blurry, or irregular damage regions. In this paper, we propose a Mask-Guided Multi-Scale Edge-Enhanced Segment Anything Model (MMESAM) for ancient mural damage detection. We uses Segment Anything Model (SAM) as the backbone, with SAM-ViT-B/16 as the pre-trained block. We aims to improve the generalization capability and detection accuracy of the proposed model. Firstly, we design a Feature-Guided Multi-scale Adapter (FGMSA) that allows SAM to extract multiscale information and meanwhile use minimal parameters. Then we introduce a Multi-Stage Feature Fusion Module (MSFM) that can integrate features across encoder layers. Next, we utilize an Edge Enhancement Module (EEM) to incorporate SAM with foreground features and edge details. We conduct the experiments on two self-built datasets of the Mogao Grottoes murals of Dunhuang and the ethnic minority murals of Yunnan. Experimental results show that our model can accurately detect the damaged regions in ancient murals, and outperforms existing approaches in both subjective visual quality and objective evaluation metrics. The EM scores of our model on the two datasets can reach 0.927 and 0.756, respectively, both significantly higher than other comparison approaches.