SoftCut: A Fully Differentiable Relaxed Graph Cut Approach for Deep Learning Image Segmentation

Graph cut algorithms can produce consistent high-quality image segmentation masks by minimizing a predefined energy function over pixels. However, defining such a function is often impracticable, especially when it comes to semantic segmentation where pixel values must convey information about the class of a pixel. On the other hand, convolutional neural networks, like U-Net, can learn to implicitly extract meaningful information from an image, but they lack explicit constraints, leading to potential rugged boundaries in the produced masks. In recent years, many solutions have been proposed to implement graph-cut algorithms into a neural network layer, and thus combine the best of both worlds, but all lack in speed or quality of the results. SoftCut, the approach proposed in this work, is a differentiable relaxation of the graph cut problem, equivalent to an intuitive electric circuit, that, used as an output activation function, is shown to outperform both U-Net and submodular optimization in terms of IoU on real-world images taken from Cityscapes, while being faster than the latter.