As autonomous systems such as drone swarms become increasingly crucial in complex missions, it is essential to ensure effective human oversight and explainable human-machine collaboration. We propose to integrate large language models (LLMs) as an interface with artificial intelligence (AI) agents to enhance explainability and incorporate human-in-the-loop control. We discuss how LLMs can bridge the gap between the technical complexities of AI-based autonomous systems and post-hoc interpretation techniques. LLM as an interface can enhance human-machine collaboration and control and increase trust and safety. Moreover, it has the potential to provide emergent capabilities and enhanced meta-learning. In our preliminary work, we integrate LLMs with deep reinforcement learning (DRL) to enhance anti-jamming capabilities of autonomous systems. Through a detailed case study, we demonstrate how this approach not only mitigates jamming threats but also facilitates human-in-the-loop control, enabling dynamic adjustments to mission parameters. Additionally, the natural language interface provided by LLMs enhances communication efficiency between human operators and drone swarms, ensuring seamless collaboration and improved operational resilience.