Cross-modality facial attribute translation with feature space consistency between imbalanced domains

Modern facial attribute translation algorithms frequently struggle with imbalanced domains, where the target domain lacks richness and diversity. To address this challenge, we propose a novel unsupervised facial attribute translation network called BALAFT. Our approach converts the task between two imbalanced domains into a multi-modality translation problem by harnessing the potential modalities within richer domains. Specifically, we analyze the source domain and learn to decompose it into a set of potential modalities without any supervision. Then we can perform a large number of balanced cross-domain translations between these all modalities as well as the target domain. Unlike traditional methods that tend to produce abstract results, our approach provides more detailed and accurate translations. Under this condition, the modalities of images in the test set cannot exactly correspond to each other in the training set. Therefore, in the inference process, we select the two most similar modalities by calculating the euclidean distance between the clustering centroids of every two modalities in the training and test sets, respectively, and define them as the same modality. This step ensures consistency in the boundary distributions of corresponding modalities across domains, further enhancing the quality of translations. In addition, to ensure the quality of the generated faces and maintain their identity invariance simultaneously, we incorporate LPIPS perceptual loss and identity ID loss into our model. Extensive subjective and objective experiments show that our method can greatly improve the quality of facial attribute translation between imbalanced domains.