Large Language Models (LLMs) have demonstrated impressive capabilities across a variety of tasks, ranging from question answering to mathematical reasoning. However, challenges such as hallucinated content and outdated parametric knowledge limit their broader applicability. Current solutions like continual learning and knowledge editing face significant drawbacks, including catastrophic forgetting and high resource demands. Inspired by how humans utilize external tools and self-reflection to improve task performance, we propose a framework called Self-Improvement. The framework iteratively refines LLM outputs using self-reflection and external tools. The process involves generating an initial response, reflecting it to identify potential flaws, leveraging tools for verification, and revising the answer based on the feedback. Experiments on free-form question answering and mathematical program synthesis demonstrate that Self-Improvement significantly enhances LLM performance, achieving more accurate and reliable results. Our findings underscore the importance of combining self-reflection with external tools to address the inherent limitations of LLMs and advance their capabilities.