GANtruth – a regularization method for unsupervised image-to-image translation

Abstract: In this work, we propose a novel and effective method for constraining the output space of the ill-posed problem of unsupervised image-to-image translation. We make the assumption that the environment of the source domain is known, and we propose to explicitly enforce preservation of the ground-truth labels on the images translated from the source to the target domain. We run empirical experiments on preserving information such as semantic segmentation and disparity and show evidence that our method achieves improved performance over the baseline model UNIT on translating images from SYNTHIA to Cityscapes. The generated images are perceived as more realistic in human surveys and have reduced errors when using them as adapted images in the domain adaptation scenario. Moreover, the underlying ground-truth preservation assumption is complementary to alternative approaches and by combining it with the UNIT framework, we improve the results even further.