Improving visual differentiation of drones and birds in aerial surveillance using trajectory features

Detecting malicious drones using aerial surveillance cameras is challenging when the distance is large, because the drone then occupies only a few pixels. Current optical detection methods rely mostly on visual appearance features. Hence, they struggle to differentiate drones from other flying objects, especially birds, when the apparent object size is small. Fortunately, the observed trajectory over time can help improve the differentiation accuracy. Here, we propose to combine classification neural networks of the object’s trajectory features and visual appearance features. We train and test the networks using our dataset containing infrared videos of drones and birds, where the variation of drone configurations and flight patterns is relatively larger than other publicly available datasets. We show that, particularly for small objects with high motion, the inclusion of trajectory features for visual classification achieves up to 22% higher frame-wise classification accuracy compared to when only visual appearance features are used. We further demonstrate that integrating both feature types provides improved accuracy over all of the considered trajectories, with 4% more of the trajectories being classified correctly. Consistent results are also shown on an open dataset, confirming the generalizability. Our study demonstrates the crucial role of information beyond the frame-wise visual appearance features in extending the operational range of aerial surveillance cameras.