In this extended abstract, we introduce a research proposal for a novel learning-based approach to achieve adaptive and resilient swarming of Unmanned Aerial Vehicle (UAV) systems through the integration of attention mechanisms and reinforcement learning. We base it upon our expertise in Reinforcement learning for agile flight in real environments and recent extensions towards learned agile swarming. Our proposed approach will leverage the Transformer network, Variational Autoencoders (VAEs), and Reinforcement Learning (RL) to create an intelligent system capable of real-time adaptation and possible in-flight learning to accommodate reactions to dynamic environments and flight anomalies. The Transformer model processes and analyzes sensor data across the swarm, capturing long-range temporal dependencies to facilitate decision-making and coordination. VAEs provide a compact latent representation of UAV state information, enabling both local anomaly detection and efficient data transmission to edge computing platforms. The RL component learns navigation policies that adapt to the current state of the swarm, guided by insights from both the VAE and Transformer outputs. By combining these technologies, we will achieve a resilient UAV swarm capable of maintaining mission integrity and operational safety even under challenging conditions such as adversarial attacks, communication signal jamming, GPS spoofing, and similar. This research is particularly relevant to participants of the GENZERO workshop, focusing on the intersection of advanced machine learning and autonomous systems.