In dynamic environments, safe motion planning for robots is one of the frequently encountered challenges in real applications. To address this issue, this paper proposes a real-time obstacle avoidance and pathfinding controller (ROAPC) for robot manipulators using a combination of deep reinforcement learning (DRL) and artificial potential field (APF) methods. Specifically, the soft actor-critic (SAC) algorithm coupled with the hindsight experience replay (HER) algorithm is developed to address sparse reward issues and enhance control performance, especially in tasks with multiple target points. In this approach, the reward function in DRL is computed based on the attraction and repulsion forces of the robot’s current position calculated using the APF method, where attraction is considered as positive reward and repulsion as negative reward, with the sum of both forces representing the reward at the robot’s current position. This method effectively guides the robot during exploration, significantly improving convergence speed. The effectiveness of the proposed method is verified by evaluating the real-time obstacle avoidance and path planning tasks of a six-degree-of-freedom robot manipulator in a simulation environment.