Space-based laser debris removal is considered to be the most promising active debris removal (ADR) method. Recent studies showed the potential feasibility of mounting a laser system on a small spacecraft for debris removal. After the spacecraft approaches the target debris, pulsed laser ablation is used to deorbit the debris. This paper presents motion planning for the spacecraft as it follows and approaches the target debris using the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm in the deep reinforcement learning framework. The motion of the spacecraft and the target debris are described in the LVLH frame. The debris approaching process is modeled as Markov Decision Processes (MDPs). Numerical simulation showed a high success rate of debris approaching. Satisfactory results were obtained with different initial settings, indicating a certain generality of the proposed approach. This preliminary work showed that reinforcement learning approaches may be an alternative in spacecraft motion planning.