Encoder stealing attacks pose a new challenge for ensuring the protection of security and safety of those models. Malicious users aim to replicate the target encoder’s functionality by training a stolen encoder with features returned by querying the target encoder. Recent attacks have successfully trained high-performance stolen encoders by aligning predictions between the target and stolen models. However, the cost of queries, a crucial factor in the success of model stealing, has not received adequate attention. In this paper, we propose a query-efficient stealing attack against image encoders. The main idea is to use a small number of query responses as external knowledge to enhance the self-learning ability during stolen model training. We also introduce a learnable perturbator on top of the stolen encoder to perturb the output features of the stolen encoder, further improving its robustness and generalization. Linear evaluation results on image datasets of various settings show that the proposed method achieves comparable or even better performance compared to existing attacks with only 25% of the query cost.