Few-shot Named Entity Recognition (NER) aims to identify entities with limited labeled samples, which is challenging in real-world applications due to the scarcity of labeled data. Traditional approaches heavily rely on pre-training and fine-tuning paradigms, ignoring the inference potential of language models. To address above challenges, we propose a novel method, In-context learning with implicit validation inference for few-shot NER (IVCoTNER), which combines the strengths of In-context Learning (ICL) and Chain-of-Thought (CoT) reasoning. Specifically, IVCoTNER guides language models through task instructions, formats, and demonstrations, while incorporating reasoning errors into CoT for better recognition. Extensive experimental results show that IVCoTNER significantly improves the performance on six public datasets, with an average improvement of 4.4% and 4.8% on 1-shot and 5-shot settings, respectively.