Training Generative Adversarial Networks (GAN) for high-quality image generation often requires large datasets. Adapting the entire network architecture through pruning during training has recently emerged as a significant advancement for data-efficient GAN, however, it can lead to suboptimal results, especially given the competing objectives of the generator (G) and discriminator (D). Addressing this, we propose LS-GAN that dynamically tailors the GAN’s architecture at a granular, layer-specific level, enabling the generation of high-quality samples even in data-limited scenarios. Specifically, larger layers are pruned first than smaller ones to preserve key features at G and D, maintaining the balance between them. Furthermore, we propose a sequential layer-wise training strategy in which a layer is pruned and then the whole model is retrained to allow the model to recover key features potentially impacted by the pruning. Our results show LS-GAN’s better performance across widely used datasets and levels of data scarcity.