Multi-focal semantic consistency assignment for category-sensitive information access
摘要
Unsupervised Few-Shot Image Classification (UFSIC) aims to alleviate the reliance on annotations and utilize coarse-grained category-insensitive semantics to distinguish novel class at training time. However, on one hand, most of the extracted category-insensitive information tends to be specific to a particular classification task, which leads to a lack of meaningful semantic information in new tasks. On the other hand, there is an over-reliance on instance invariance for disentangling fine-grained semantics. To address these issues, we propose Multi-Focal Semantic Consistency Assignment (MFSCA), a framework for making connection between instance invariance and partition variance, which can further mine category-sensitive semantic information in samples for solving novel classification tasks. Particularly, instance-level prototypes are constructed by contrastive learning to obtain instance-invariant representations. As instance-invariant representations lack category-sensitive information, powerful clustering-level prototypes are constructed dynamically by partition-based deep clustering to disentangle fine-grained semantics. Regarding this, our approach rises from focusing a single-focal point for specific task to multi-focal points for general tasks, enabling the extraction of more comprehensive representations. It alleviates the problem of losing intra-instance semantic information caused by focusing only on coarse-grained semantics, and classifies images of complex scenes effectively. Extensive experiments on miniImageNet and tieredImageNet datasets demonstrate the effectiveness of our MFSCA.