Deep Neural Networks (DNNs) are widely used in Unmanned Aerial Vehicle (UAV) applications such as post-disaster rescue, urban surveillance and so on. However, the increasing complexity of DNN models and their high computational demands pose significant challenges for their deployment on resource-constrained UAVs. Existing solutions mainly focus on two approaches: one is to deploy lightweight DNN models on UAVs for local inference, which provides low latency but inevitably leads to significant accuracy loss; the other one is to conduct the collaborative inference of complex DNN models between UAVs and the resourceful ground station, which maintains high accuracy but still results in relatively large latency under dynamic and limited air-ground communication resources. Integrating the two aforementioned approaches could potentially achieve both high accuracy and low latency, which however remains unexplored so far. This paper studies how to adaptively switch between lightweight and complex DNN models, to boost a fine tradeoff between accuracy and latency in an air-ground system. Specifically, we jointly optimize the decisions of model selection and partition to minimize the overall latency of a DNN task flow, under the UAV memory constraint and expected accuracy requirement. Given the combinatorial explosion nature of this problem, leveraging Deep Reinforcement Learning (DRL) techniques, we propose an Adaptive Switching algorithm to effectively switch between Lightweight and Complex DNN models (ASLC). Extensive simulations based on real-world UAV movement trajectories demonstrate that the ASLC algorithm consistently outperforms three benchmark algorithms, achieving the lowest latency while always satisfying the accuracy requirement.