Large language models (LLMs) struggle with event detection (ED) due to the structured and variable number of events in the output. Existing supervised approaches rely on a large amount of manually annotated corpora, facing challenges in practice when event types are diverse and the annotated data is scarce. We propose Generate-then-Revise (GtR), a framework that leverages LLMs in the opposite direction to address these challenges in ED. GtR utilizes an LLM to generate high-quality training data in three stages, including a novel data revision step to minimize noise in the synthetic data. The generated data is then used to train a smaller model for evaluation. Our approach demonstrates significant improvements on the low-resource ED. We further analyze the generated data, highlighting the potential of synthetic data generation for enhancing ED performance.