Dense Residual Connections in Generative Adversarial Networks
摘要
We investigate the effect of dense residual connections in generative adversarial networks up to 20,000 epochs. Each consequent block layer receives all previous outputs, which are joined by an identity mapping before applying an activation function. In addition, we investigate the effect of using real label smoothing and label flipping of real and fake labels during the training of the discriminator. Using batch normalization layers after each convolution layer, in the discriminator or the generator, results in model instability leading to convergence failure or mode collapse which never manages to recover image generation. In the experimental runs that don’t use batch normalization layers, there is evidence of mode collapse and checkerboard artifacts early in the training which disappears later in the training. Label flipping in the discriminator results in a model that stabilizes and recovers image generation earlier compared to the other models (epoch 2750; baseline model with no residual connections—epoch 5000, model with residual connections—8000, with label smoothing—4500). Overall, generative adversarial networks with dense residual connections don’t show much improvement over the baseline model when it comes to stable training. The training of the dense residual models requires about 4 times as long to train.