In the real world, it is hard to extract large data with the characteristics that are labeled. This becomes a problem when trying to implement artificial intelligence to real-life problems. In this work, we propose a self-attention siamese network guided by the principles of unsupervised learning and few-shot learning which can learn robust representation without requiring the large number of labeled data. Our approach introduces a self-attention module to capture rich contextual information and a projection head that embeds features into a discriminative space. We have performed a comprehensive experiment evaluation to show the effectiveness of our approach compared to other supervised training methods and other few-shot learning techniques. Our results highlight the potential of adding self-attention to the siamese network for enabling unsupervised few-shot learning tasks, paving the way for more efficient machine learning systems capable of learning from limited unlabeled data.