Q-Learning Based Framework for Solving the Stochastic E-waste Collection Problem

Electrical and Electronic Equipment (EEE) has evolved into a gateway for accessing technological innovations. However, EEE imposes substantial pressure on the environment due to the shortened life cycles. E-waste encompasses discarded EEE and its components which are no longer in use. This study focuses on the e-waste collection problem and models it as a Vehicle Routing Problem with a heterogeneous fleet and a multi-period planning problem with time windows as well as stochastic travel times. Two different Q-learning-based methods are designed to enhance the search procedure for finding solutions. The first method involves utilizing the state-action value to determine the order of multiple improvement operators within the GRASP framework. The second one involves a hyperheuristic that extracts a stochastic policy to select heuristic operators during the search. Computational experiments demonstrate that both methods perform competitively with state-of-the-art methods in newly-generated small-sized instances, while the performance gap widens as the size of the problem instances increases.