Large Language Models (LLMs) have advanced the automation of code generation, yet their reliability remains limited when the examples provided in context are noisy or inconsistent. In practice, real-world code data are rarely clean or complete; they often contain partial implementations, redundant logic or is often messy that can reduce model performance. To overcome these limitations, recent work has explored the use of synthetic data generated to supplement scarce or imperfect real data. While this strategy can increase diversity and coverage, it also raises the risk of introducing unreliable or misleading patterns, leading to reduced accuracy in complex generation tasks. This doctoral research investigates how to make LLM-based code generation more reliable when synthetic data are incorporated into the in-context learning process. By interpreting code generation through the lens of Information Retrieval (IR), the study examines how relevance, coverage, and trustworthiness can be modeled to evaluate synthetic examples and mitigate hallucinations. The overarching goal is to build a reliability-aware framework that ensures synthetic contexts consistently strengthen code generation performance.