<p>We provide a dataset for object detection and tracking in aerial imagery, namely “M3OT”. M3OT is a multi-modality vehicle detection and tracking dataset acquired by two Unmanned Aerial Vehicles (UAVs) in a high-altitude region, consisting both RGB and infrared thermal (IR) modalities, ranging from 100 m to 120 m. Owing to the high-altitude acquisition, the annotated objects in the dataset are predominantly small objects, which poses significant challenges for detection and tracking algorithms. The dataset comprises 21,580 frames extracted from 8-hour videos, including 10,790 paired RGB-IR images, with a total of 220,000 bounding boxes annotated across various environments such as suburban, urban, daytime, dusk, and night. To our knowledge, this is the first multi-drone multi-modality dataset designed for multiple object tracking. We evaluate state-of-the-art multiple object tracking algorithms on this dataset. The experimental results indicate that the M3OT dataset presents a challenging benchmark for multiple object tracking. We believe that the M3OT dataset can contribute to applications and research based on vehicle detection and tracking from a UAV perspective. The dataset is freely available at <a href="https://figshare.com/s/01fa8d1163f4e9a5a13a">https://figshare.com/s/01fa8d1163f4e9a5a13a</a></p>