Clustering of Face Images in Video by Using Deep Learning

In deep learning research, self-supervised learning is one of the recent hot topics, and it is not necessary to prepare labels for the training sample. In this study, we would like to consider how to train the model parameters without explicit training labels from video. As an example task, we consider the problem of understanding the person’s identity in the video based on the face images captured from the video by using face detection and face tracking. The face images are not annotated with the person’s identity, but we can notice that the face images in the tracked sequence are the same person. Also, the face images in the different sequences in the same frame must be different persons. In this paper, we develop an automatic clustering method of face images in the video using these clues. The paper introduces two techniques: the incorporation of Center loss into the loss function and the addition of Auto Encoder structure to the network. Including Center loss ensures that tracked groups of face images (Track) have features closer to the average features of their respective Track, resulting in features invariant to facial variations and noise. Additionally, the addition of AutoEncoder structure aids in dimensionality reduction, enabling the extraction of high-quality features. The combination of these methods yields improved clustering accuracy compared to prior research. Furthermore, this paper explores approaches related to Online Clustering, which relaxes the constraints on determining the number of clusters. This alternative approach achieves comparable accuracy without the need for precise determination of the cluster count.