Approaches to Face Verification Through Attribute-Based Attention

Face recognition as a domain aims to generate salient representations of face images. However, a face image has several facial attributes, such as eyeglasses, facial hair, etc. We hypothesize that leveraging these attributes and thus making the matching process agnostic to these attributes should lead to better results. The black-box nature of typical CNN based single Face Recognition DNN’s embedding doesn’t allow leveraging the concept of face-attributes. We have therefore devised the following methods. Feature-level suppression prior to matching - Here Vision Transformers, allows us to assign zero weights to attention maps at specific spatial regions where an attribute that we wish to be agnostic of, is usually present. We have demonstrated improvement in verification accuracy for the face-masks attribute by over 1% EER and 0.6% TAR@FAR1-e4. Aside from this approach, we also show if we trained a face attribute detector over a face-recognition backbone network, then the final layer embedding allows us to suppress specific neurons highly correlated with an attribute. Score-level suppression: Here, for a given facial attribute, we suppress the relevance of the attribute by binning the scores of all probes and gallery pairs exhibiting similar face attribute configuration i.e. whether a probe and gallery pain possess a facial attribute or not, and then find a unique threshold to operate the matching process on. Finally, all the above methods mandate that we detect the facial attribute first prior to matching. We have investigated novel ways to effectively and efficiently train such face attribute detectors.