Differentiable Largest Connected Component Layer for Image Matting

In the image matting task, a foreground object typically corresponds to a connected component in an image. Recent work has extracted the largest connected component in the raw object alpha matte to form the cleaned object alpha matte. However, existing works only consider connected components after training. We argue that including it in training can introduce prior knowledge that the object corresponds to a whole region, which can further improve the model’s performance. To achieve this, we implement a differentiable largest connected component (LCC) layer, which finds the largest connected component and calculates the backward propagation result for the gradient. This LCC layer can be added to a model just like other PyTorch layers. Furthermore, we designed a gradient backward propagation method for our LCC layer, which facilitates the model to learn to output foreground pixels as a connected component. Its effectiveness has been comprehensively validated on different models and datasets. Finally, although our LCC layer significantly improves the models’ performance on average on general datasets, we further conduct a study of which sorts of objects in the image matting task will benefit from the LCC layer, to give guidance on when our LCC layer should be used. The code is available at https://github.com/XinshuangL/LCCLayer .