This research introduces a resilient cryptographic architecture that integrates Hyperelliptic Curve Cryptography (HECC) with adaptive key management, enhanced through Particle Swarm Optimization (PSO), to safeguard Unmanned Aerial Vehicle (UAV) communications. The system utilizes the Jacobian group of a genus-2 hyperelliptic curve over a finite field \( \mathbb {F}_p \) (where \( p \equiv 5\, \text {mod}\,{8} \) , a 256-bit prime) to perform a secure Diffie-Hellman key exchange, generating a shared secret via SHA3-256, in conjunction with AES encryption for effective and robust message safeguarding. PSO optimizes parameters like private key length, AES key length up to 32 bytes, and key rotation intervals, balancing key generation duration with entropy maximization. The proposed framework adaptively modifies AES key lengths according to variable UAV network conditions and changing security requirements, while an advanced key rotation manager for AES reduces vulnerabilities by regularly renewing keys based on a thorough assessment of attack vectors, such as Man-in-the-Middle, Physical Capture, Interception, Key Reuse, and Brute-Force. This versatility guarantees customized security for resource-limited UAV systems, facilitating safe bidirectional communication between Ground Control Stations (GCS) and UAVs in critical conditions. Experimental findings demonstrate substantial enhancements in efficiency and security, characterized by improved entropy and robust resistance to cryptographic assaults, rendering this framework a versatile solution for modern UAV communication security issues.