Closing the Sim-to-Real Gap: Enhancing Autonomous Precision Landing of UAVs with Detection-Informed Deep Reinforcement Learning

Autonomous precision landing of a UAV is a challenging task relying on simultaneous target localization and control. It can not be performed using GPS coordinates, due to limitations in the accuracy of the technology. Control policies are fine-tuned in simulation before deployment but most simulators are equipped with low-quality graphics, posing a challenge when utilizing vision based algorithms, intended to be implemented in the real world. In this paper we showcase a joint computer vision and reinforcement learning approach, in the photo-realistic simulator AirSim, to reduce the sim-to-real gap. Localization is performed using the Yolov8 object detector and control using the PPO and LSTM-PPO algorithms. We achieve a 94.25% and 94.86% success rate, over 1391 and 817 landings at three different simulated environments, respectively.