<p>Reinforcement learning (RL) is effective in complex manipulation tasks; however, training RL policies in obstacle-rich environments remains challenging. Most existing RL models require very deep networks to encode obstacle information from high-dimensional point cloud data and rely on sparse collision penalties, resulting in inefficient learning. To address this limitation, this study proposes an obstacle-aware RL framework that introduces additional states and rewards derived from potential field-based repulsive forces. Because the proposed representation encodes the geometric relationships between the robot and obstacles using a fixed number of state variables, regardless of the number or shape of obstacles, it mitigates the learning instability caused by high-dimensional point cloud inputs. Experimental results demonstrate that the proposed representation is sensitive to the relative spatial relationship between robots and obstacles. Furthermore, the proposed model achieved a success rate of over 81.1% across various unseen environments, demonstrating strong generalization performance. In complex tasks such as opening a door with and without a lever, the proposed method achieved success rates of 65.0% and 75.3%, respectively, without collisions. These results demonstrate the effectiveness of repulsive force-based states and rewards in enabling collision-free RL manipulation in complex scenarios.</p>