SAYNet: Self-attention YOLO Network for Human Detection in Thermal and Infrared Images

Multi-object detection in a video sequence is a significant challenge for effective surveillance in environments where varying lighting and other environmental factors constrain vision in different places. In this regard, the use of traditional visible cameras (VC) may significantly degrade the performance of multi-object detection as it is sensitive to a variety of environmental factors. In this work, we proposed a self-attention guided deep learning method to effectively detect multiple human objects (MHO) in Thermal (Tr) and Infrared (Ir) video sequences. In our work, we incorporated Self-Attention Block (SAB) in the You Only Look Once (YOLOv8) model, coined as Self-Attention YOLO Network (SAYNet), to detect MHO in Tr and Ir video sequences more effectively. The efficacy of SAYNet has been analyzed on four standard Tr and Ir datasets, namely CVC-09, CVC-14, UNIRI-TID, and LLVIP, concerning various performance measures. From the experimental results, it is found that the use of the self-attention in the YOLO model, i.e., SAYNet, performs significantly better than different variants of the YOLO model.