The material delivery route prediction problem aims to predict the future delivery routes of couriers given a pool of delivery tasks. This task faces significant challenges due to the highly nonlinear and complex nature of a large amount of historical data, as well as factors related to individual courier preferences. Most existing methods utilize deep neural networks based on supervised learning to capture behavioral patterns from massive historical data. However, these methods often exhibit limitations in handling the dynamic nature of data and the diversity of individual preferences. This paper proposes a novel deep reinforcement learning framework that integrates a Variational Autoencoder (VAE) to handle large-scale data features and introduces dynamic embedding features to accurately reflect individual preferences. The framework uses Proximal Policy Optimization (PPO) for training to achieve an optimized strategy. Validation on two publicly available real-city delivery datasets provided by Cainiao as well as on multiple national city delivery datasets generated by ourselves, demonstrate that the predictive performance of this framework significantly outperforms seven existing prediction methods.