Enhancing GAN Compression by Image Probability Distribution Distillation

This paper presents a novel approach named Image Probability Distribution Distillation (IPDD) for compressing generative adversarial networks (GANs) by distilling knowledge from the global distribution of images. Unlike traditional methods that distill at the pixel level, we propose a holistic approach that captures the overall coherence of images. To achieve this, we introduce a novel teacher discriminator that engages in adversarial training with both the teacher generator and the student generator in asynchronous weighted manner, using variable weights to optimize the Nash equilibrium between them. Our framework explores the uncharted territory of mining global distribution information and federated training of the teacher discriminator, offering potential for enhancing the performance of compressed GANs. Extensive experiments on benchmark datasets demonstrate that our approach significantly reduces GAN complexity while achieving optimal performance.