Onfire 2023 Contest: what did we learn about real time fire detection from cameras?

Several methods for fire detection from camera streams have been proposed in recent years. While traditional techniques often emphasize recall, they frequently neglect critical factors such as minimizing false positives, ensuring timely alarm notifications and optimizing performance for devices with limited computational resources. The ONFIRE 2023 contest evaluates various approaches for detecting fire using smart cameras and establishes new evaluation metrics to measure precision, recall, notification promptness, processing speed and resource utilization. The eight participating teams received a training set that integrated all publicly available video datasets and were evaluated on a private test set. The latter includes positive samples where fire is not present at the beginning of the clip, as well as negative samples featuring moving fire-like objects. In this paper, we provide an overview of the competition’s dataset and review the proposed solutions, highlighting the winning approach, the limitations of existing datasets and the evaluation metrics used. By analyzing the results of the competition, we propose possible design choices and future directions that may help to reduce the false positive rate while preserving accuracy.