Temporal Dependency-Oriented Deep In-Loop Filter for VVC

Video coding schemes exploit the temporal correlation between the video frames for high coding efficiency. Thus, the coding efficiency of one frame is dependent on the quality of its reference frames. Such temporal dependency is seldom considered in the existing in-loop filters, since they optimize the current frame reconstruction quality but ignore its effect on the subsequent frames. We address this limitation and propose a temporal dependency-oriented deep in-loop filter, namely TDOF. First, we design a deep network that uses several kinds of guiding information to help the network distinguish different regions of different temporal dependency importance. Second, we train the network using the combination of two losses to optimize not only the current frame reconstruction quality but also the next frame prediction quality. We implement the TDOF with a simple network structure for the Versatile Video Coding (VVC) standard. Our method achieves on average 1.77% BD-rate reduction compared to VTM-10.0 under the Low-delay P configuration. Ablation studies demonstrate the effectiveness of the proposed method.