Infrared small target detection is of vital importance yet extremely challenging in anti-UAV operations against complex backgrounds. The targets are so minuscule and faint that they are hard to be identified by conventional methods, which primarily rely on spatial appearance features. To tackle this problem, inspired by biological visual systems that mainly use motion cues to search for targets, we propose an Inter-Frame Motion-Aware network (FMA-Det) for anti-UAV small target detection. This approach comprises three key components: a temporal auxiliary branch equipped with a motion-aware module that leverages inter-frame information to enhance the awareness of target motion; a spatiotemporal feature fusion mechanism that integrates background-suppressed temporal information with spatial features from the main branch, thereby improving target feature extraction; and a small-scale multilevel auxiliary mechanism that fuses shallow and deep features to increase sensitivity to small targets. Extensive experimental results demonstrate that FMA-Det achieves state-of-the-art performance on large-scale anti-UAV datasets, significantly boosting the accuracy of detecting small targets against complex backgrounds. Particularly, compared to other advanced competing methods, the AP50 value increases by 5.9% when the targets are less than 3% of the image size. The source codes with results will be publicly available.