Using LLMs to Manage Voice-Based Aerial-Ground Communication in UAV Systems

Natural language-based control commands for large-scale UAVs hold promise in applications such as voice-centric air traffic control (ATC) environments, single-operator management of multiple UAVs, and eVTOL taxi services. Large language models (LLMs) can enable command translation, knowledge base queries, and contextual understanding of diverse phrasings, while providing operational recommendations. However, challenges including speech recognition errors, harmful command inputs, and timeout issues hinder practical deployment. This paper proposes an LLM-based UAV interaction agent designed to parse commands from both UAV cockpits and ATC systems, while generating actionable operational advice. We evaluated multiple LLM variants and implemented engineering optimizations to improve recognition accuracy and response speed. Surveys with UAV stakeholders indicate that LLM-driven systems partially meet operational requirements but underscore the need for improvements in training methodologies and accuracy. These findings highlight the potential of LLMs in onboard UAV communication while emphasizing unresolved technical challenges requiring further refinement.