<p>The assist-as-needed control paradigm is a widely employed strategy in physical human-robot interaction, particularly in assistive and rehabilitation-oriented training tasks. However, most existing approaches rely either on accurate estimation of the torques generated by the user or on predefined trajectories, which may require extensive parameter tuning or limit the natural variability of movement, respectively. This paper presents a novel assist-as-needed control framework that integrates reinforcement learning with a fuzzy supervisor and a force field-based tunnel for upper-limb three-dimensional reaching tasks. A fuzzy logic-based reward function guides training of the reinforcement learning agent, which learns to adjust Cartesian impedance without relying on predefined trajectories. A high-level fuzzy supervisor measures directional similarity between instantaneous movement direction and a heuristic reference approximation of the target vector and activates a force field-based tunnel mechanism when the similarity falls below a threshold, correcting the path or even assisting the user in reaching the target. The framework was evaluated using three actor-critic agents on a KUKA LBR iiwa with six healthy subjects. In simulation, the reinforcement learning policy achieved over 85% impedance-adherence rate, while the fuzzy supervisor and force field-based tunnel achieved close to 100% triggering consistency. In real-world tests, the reinforcement learning policy achieved approximately 80% impedance adherence, and the fuzzy supervisor and force field-based tunnel exceeded 90% triggering consistency. These results demonstrate the feasibility of the proposed adaptive impedance modulation and supervisory assistance architecture in healthy-subject human-robot interaction. Clinical validation in post-stroke rehabilitation settings remains an important direction for future work.</p>