Image Dehazing Based on Online Distillation

Advanced supervised single-image dehazing models require a large number of trainable parameters and a huge amount of training data, containing a paired set of hazy images and corresponding clear images. To address this, knowledge distillation paves the way for training a small student network with the help of a larger teacher network. We propose an online distillation network for image dehazing in which, the teacher is an autoencoder network with feature attention blocks, and the student is a smaller autoencoder with fewer feature attention blocks. Specifically, the proposed model trains both the heavy Teacher network and the compact student network at the same time, with the student network learning the weights of the intermediate layers from the teacher network. The results of the experiments conducted on both indoor and outdoor datasets, demonstrate a significant improvement in performance compared to the state-of-the-art models on the basis of both image quality and fewer model parameters.