In this work, an intelligent assembly algorithm with a force-velocity controller is presented, which is designed to solve the peg-in-hole problem using lithium-ion cells as an example. In addition to the controller, a reinforcement learning trained Twin Delayed Deep Deterministic Policy Gradient (TD3) agent with recurrent neuronal networks (RNN) is used to assemble lithium-ion cells into holes with uncertainties and interfering contours. Particularly in the context of small-batch production, such assembly algorithms can offer significant advantages, as they allow for a flexible response to tolerances and interfering contours, which is currently still largely handled by hand. This contribution presents initial approaches and results towards the development of such an assembly algorithm.