FPGA-accelerated real-time DCGANs via Xilinx DPUs and Vitis AI

Generative adversarial networks (GANs) produce high-quality images but are computationally intensive, especially due to transposed-convolution operations, limiting their real-time performance on traditional hardware. To address this, we propose an optimized FPGA-based acceleration framework leveraging AMD deep learning processing units (DPUs) and the Vitis AI toolchain to enable real-time inference of deep convolutional GANs (DCGANs) for image reconstruction. The proposed approach applies a two-stage quantization method that profiles layer-wise dynamic ranges and fine-tunes scale factors via host-side retraining. This enables quantization of both generator and discriminator from 32-bit floating-point to INT8 precision with minimal accuracy degradation. Additionally, structured pruning through the Vitis AI Optimizer removes redundant weights and filters, producing a compact model whose weights can reside in on-chip memory during layer execution, thereby improving DPU efficiency. The architecture uses a multithreaded ARM processor to manage preprocessing and DMA operations, while a lightweight scheduler in programmable logic sequences the execution of convolution kernels across multiple DPU cores. Double buffering is employed to overlap data movement with computation. On the Zynq UltraScale+ MPSoC ZCU104, the proposed INT8 quantized accelerator achieves real-time inference at 105 FPS on the FMNIST dataset and 47.9 FPS on PlantVillage. Relative to the CPU FP32 baseline, this corresponds to up to 3.5 \(\times \) $\times$ and 2.9 \(\times \) $\times$ higher throughput, respectively, while remaining comparable to the GPU INT8 throughput. Using nominal power references rather than runtime-measured energy, the corresponding FPS/W estimates are up to 7.3 \(\times \) $\times$ and 6.2 \(\times \) $\times$ higher than the CPU FP32 baseline. It maintains FID within roughly 5 points of the corresponding full-precision models, indicating that the quantization-and-pruning pipeline preserves competitive image quality for edge-oriented GAN deployment.