Industrial-SAM with Interactive Adapter

Image segmentation is a fundamental task in computer vision that aims to partition an image into multiple regions or objects. The recently proposed Segment Anything Model (SAM) is a powerful and promptable model for image segmentation that can produce high-quality object masks from input prompts such as points or boxes. However, recent studies have found that SAM, which is trained on natural images, performs poorly on non-natural images, such as industrial images, where the objects and backgrounds may have different characteristics and distributions. To address this problem, we propose Industrial-SAM, a SAM adapted for industrial vision applications. Industrial-SAM leverages the pre-trained SAM as a backbone and fine-tunes it on domain-specific datasets of industrial images. To efficiently train Industrial-SAM and make it suitable for industrial vision, we design an Interactive Adapter. Interactive Adapter is a novel module that can incorporate human feedback during training to guide the model learning. It can also generate adaptive prompts for different images based on their content and difficulty. We conduct extensive experiments on the cap welding segmentation dataset and seal pin welding segmentation dataset. The experimental results show that Industrial-SAM can train excellent industrial vision models with the help of human feedback.