Spatial Localization and Detection of Heat Source by Fusing Thermal Images with Point Clouds

Searching for heat sources in dim or foggy environments is an important task in firefighting or rescue operations. To accurately detect and localize heat sources, a method by fusing 3D point clouds with 2D thermal images is presented. Firstly, the intrinsic parameters of the thermal imaging camera are estimated by using a self-made calibration board. And the extrinsic calibration of solid-state lidar and thermal imaging camera is performed to establish a geometric relationship between them. Then, the 2D image frames are time-synchronized with the 3D point cloud frames under the framework of the robot operating system (ROS). After fusion, the point clouds contain both the depth information and the colors (corresponding to the temperature of the surrounding objects). Heat source is detected based on the thermal images and its location can be obtained by mapping from the camera coordinate frame to the lidar coordinate frame. Experiments have verified that, even in complex environments, our method can provide the heat source location in three-dimensional space with high accuracy.