<p>Complex tasks on vehicles might have overdue and energy issues which are caused by the limited local computing resources. Mobile Edge Computing (MEC) provides support for task offloading to reduce completion time and energy consumption. Yet MEC servers might be overloaded with increasing tasks, which can be handled by recruiting computing resources from nearby vehicles without tasks. To offload tasks to MEC servers and task-free vehicles, a multi-hierarchy task offloading optimization method based on deep reinforcement learning is proposed. Tasks are first divided into multiple hierarchies. Then the priority values of the tasks in each hierarchy are calculated. Finally, the task offloading problem is modeled as a Markov Decision Process (MDP), and the Deep Q-Network is used to obtain a better offloading solution based on the sorted tasks for each hierarchy. Experimental results show that the proposed method can obtain a comparatively satisfactory solution.</p>