Machine Learning in the Last-Mile Delivery: Modified Q-Learning for the TSP

The increasing demands of e-commerce have increased the need for efficient last-mile delivery solutions, presenting logistical challenges such as traffic congestion and high operational costs. This paper explores the application of machine learning (ML) techniques to optimise last-mile delivery, focusing on Q-learning algorithm to solve the Travelling Salesman Problem (TSP). A case study is presented, where modified Q-learning is applied to the TSP, demonstrating its potential to optimise delivery routes and reduce costs. The proposed model is implemented in the MATLAB environment and it is tested on symmetric and asymmetric small and medium-sized instances from TSPLIB as well as on the real-life TSP instances. Experimental results indicate that ML-based model demonstrates better results on average in most instances compared to Basic Q-learning, Double Q-learning, and SARSA algorithms. This paper contributes to the growing body of knowledge on ML applications in logistics, offering insights into future research directions and practical implementations in last-mile delivery.