Over the past decade, there has been a growing trend in deploying drones (UAVs) equipped with various sensors, including RGB cameras, infrared sensors, and LiDAR, for data acquisition and analytics. Drones are used in areas such as building surveillance, industrial site security, traffic management, disaster management, and border patrol, primarily focusing on detecting specific objects in aerial images. This work aims to detect eight object categories: pedestrians, cars, buses, trucks, bicycles, tricycles, people, and motorcycles. While recent object detection efforts using deep learning rely on computation-heavy networks, drones often face limitations in flight time and onboard computational power due to their power source. Therefore, object detection tasks must be performed using lightweight networks with fewer FLOPs. This work proposes a lightweight aerial object detector inspired by YOLOv5 architecture, featuring a backbone extractor, Feature Pyramid Network, and Detector Heads. This design outperformed seven state-of-the-art methods on the VISDRONE-2019 dataset through extensive experimentation with different module combinations, achieving a mAP of 26.9% with 0.958M parameters and 1.6 GFLOPs.