Semi-supervised Task Aware Image-to-Image Translation

Training neural networks on new domains requires a huge amount of data which is often expensive to capture and annotate, in particular for tasks like semantic segmentation. In such cases, it is desirable to train a model in a synthetic counterpart domain where annotated data is virtually unlimited. However, such models often suffer under the domain gap between training (synthetic) and deployment (real). Bridging the domains with image-to-image translation methods is feasible on input level but disregards the downstream task. In this work, we review our task aware image-to-image domain adaptation approach [40]. With the help of some annotated target data, we guide a CycleGAN based image-to-image translation model to generate downstream task aware data serving as input for the task model operating on synthetic data. Our approach is modular and applicable to different tasks such as image classification and semantic segmentation. For classification, we further extended it to an active domain adaptation approach with uncertainty-based query strategy. We present numerical experiments containing evaluations on complex domain adaptation tasks, domain gap analyses based on downstream task networks trained from scratch as well as the impact of informed data selection. Our method outperforms CycleGAN by 14 percent points and supervised from scratch training on real world data by 28 percent points in accuracy in image classification using only 75 actively selected, labeled images. For semantic segmentation we achieve a performance increase of up to 12.5 percent points in mean intersection over union on Cityscapes when using up to 148 labeled images.