Using LiDAR Data as Image for AI to Recognize Objects in the Mobile Robot Operational Environment

Nowadays, there has been a growing interest in the use of mobile robots for various applications, where the analysis of the operational environment is a crucial component to conduct our special tasks or missions. The main aim of this work was to implement artificial intelligence (AI) for object detection and distance estimation navigating the developed unmanned platform in unknown environments. Conventional approaches are based on vision systems analysis using neural networks for object detection, classification, and distance estimation. Unfortunately, in the case of precise operation, the used algorithms do not provide accurate data required by platforms operators as well as autonomy subsystems. To overcome this limitation, the authors propose a novel approach using the spatial data from laser scanners supplementing the acquisition of precise information about the detected object distance in the operational environment. In this article, we introduced the application of pretrained neural network models, typically used for vision systems, in analysing flat distributions of LiDAR point cloud surfaces. To achieve our goal, we have developed software that fuses detection algorithm (based on YOLO network) to detect objects and estimate their distances using the MiDaS depth model. Initially, the accuracy of distance estimation was evaluated through video stream testing in various scenarios. Furthermore, we have incorporated data from a laser scanner into the software, enabling precise distance measurements of the detected objects. The paper provides discussion on conducted experiments, obtained results, and implementation to improve performance of the described modular mobile platform.