Large Language Models (LLMs) have been demonstrated to possess a strong ability for in-context learning (ICL), whereby adaptation to new tasks is achieved from only a few demonstrations without parameter updates. However, this capability has been found to be brittle, as models frequently overfit to superficial characteristics of provided examples—a phenomenon referred to as constraint imitation—which diminishes performance on complex reasoning tasks requiring robust generalization. To address this issue, Counter-Example Guided In-Context Learning (CE-ICL) is introduced as a novel prompting methodology in which standard few-shot prompts are augmented with contrastive examples of plausible yet flawed reasoning paths. By being exposed to common fallacies and incorrect solution strategies, a more discriminative learning signal is generated, compelling the underlying logic of tasks to be captured rather than stylistic or structural patterns merely being mimicked. Systematic empirical analysis took place across the arithmetic, commonsense and symbolic rationalization test sets all-inclusive of the GSM8K and StrategyQA using state-of-the-art LLMs including Llama-3-70B and GPT-4o. The analysis revealed CE-ICL bested standard ICL and Chain-of-Thought (CoT) prompting consistently on the accuracy of tasks while reporting statistically significant improvements in the semantic diversity of correct rationalization routes. Such insights demonstrate CE-ICL inhibits constraint imitation effectively, encourages more flexible and generalizable rationalization, and represents advances toward more robust LLM rationalization.