Vision-Based Aircraft Marshalling Recognition and UAM Control Command Generation

Urban air mobility (UAM) represents a burgeoning mode of air transportation, presently in its nascent operational phase with manned aircraft that necessitate pilot assistance. The ultimate objective for UAMs is the development of unmanned aircraft capable of autonomous flight. Ongoing research and development efforts aim to realize this objective. This thesis proposes a vision-based software framework to control UAMs by recognizing aircraft marshalling during takeoff and landing at vertiports. Specifically, this paper introduces an image processing algorithm employing deep learning-based human pose estimators and object detection to identify aircraft marshallers, ascertain the marshalling actions they perform, and translate these actions into control commands that are transmitted to the UAM. The algorithm’s validation involves experiments conducted using System in the loop (SITL) simulation, with the results subsequently analyzed. The findings indicate that the proposed software framework performs effectively in a real-time image processing environment.