MABC-Net: Multimodal Mixed Attentional Network with Balanced Class for Temporal Forgery Localization

As an impactful social task, recognizing and localizing forgery events in videos is getting active attention. Since forgery events occur in both auditory and visual modalities, detailed perception of multimodality is essential for accurate temporal forgery localization (TFL). Currently, most fake videos consist of only a small segment of fake content, which leads to the problem of class imbalance due to large differences in the proportion of fake and real content. Unfortunately, existing methods suffer significantly in performance owing to the fact that they take little account of the problem of class imbalance. To address this issue, we present a multimodal Mixed Attentional network with Balanced Class (MABC-Net) for temporal forgery localization. Specifically, we first propose the mixed-attentive feature learning (MAFL) module. This module captures audio-visual temporal features via a mixed learning strategy, which leverages two self-attention blocks and two cross-attention blocks. Moreover, a fusion-balanced localization (FBL) module is designed for alleviating the influence of the class imbalance problem. This benefits from an elegant combination of focal and boundary matching loss functions. Extensive experiments on TFL show that our MABC-Net is superior to the state-of-the-art methods and localizes more precise segment boundaries. Code is available at https://github.com/Tea7374/MABC-Net.