Fire and smoke detection in outdoor environments poses critical challenges for public safety and environmental protection. Traditional single-frame detection methods often fail to capture the temporal dynamics essential for early fire event recognition. This paper introduces Event-GPT, a novel hybrid architecture that combines VideoMAEv2 encoder with a LoRA-tuned GPT-2 decoder for temporally-aware fire and smoke event classification. Our approach processes video sequences through an iterative fusion strategy that accumulates spatiotemporal embeddings across time, enabling robust reasoning over long-term temporal dependencies. Unlike conventional frame-based classifiers, Event-GPT maintains an evolving internal state that captures the progressive nature of fire events. Extensive experiments on the ONFIRE 2025 dataset demonstrate that our method achieves 84.61% precision and 91.66% recall with an average notification delay of 7.14 s when trained at 1 FPS and tested at 4 FPS. The architecture’s memory-efficient design, enabled by LoRA adaptation, ensures real-time performance while maintaining high detection accuracy. The code associated with this manuscript is available at: https://github.com/unica-visual-intelligence-lab/Event-GPT