Progressive Adversarial Learning for Multi-modality Biometric Recognition with Missing Modality
摘要
Multimodal biometric recognition technology has garnered significant attention in recent years owing to its benefits in security and recognition precision compared to traditional single biometric recognition methods, the fusion of multiple biometric features can effectively mitigate the constraints of individual features, reducing issues such as recognition failure caused by forgery or inability to capture a single biometric trait. Given that scenarios including missing modalities may occur, most approaches primarily address missing modalities as independent tasks, ignoring the interrelationships among different combinations of missing modalities and insufficiently adapting to varied input modalities. Thus, we introduce a progressive adversarial learning framework that enables the joint utilization of various missing modality combinations. Specifically, a semantic consistency learning is developed, employing an adversarial technique for arbitrarily missing modality combinations. Additionally, a progressive modality completion learning network is utilized, incorporating arbitrary combinations of missing modalities with complete modality combinations for adversarial learning. This progressively modifies the interrelation across modality combinations, enabling the model to acquire multi-modal semantic information that resembles a complete modality combination. Extensive experiments have been performed to validate the effectiveness of the proposed method.