Reinforcement Learning (RL)-based controllers have recently gained attention as AI-driven, model-free methods for controlling power electronic converters by learning optimal control actions through continuous interaction with the environment. Their learning process is governed by a reward function, which guides the agent’s behavior. This paper investigates the influence of incorporating penalty terms into the reward function on the training efficiency and performance of an RL-based controller for a 7-level grid-tied Packed-U-Cell (PUC7) multilevel inverter. The controller is developed using the Double Deep Q-Network (DDQN) algorithm, selected for its balanced combination of strong performance and ease of implementation. The control objectives include sinusoidal current injection into the grid and capacitor voltage regulation around the desired value. The reward function is designed based on current and voltage tracking errors, with two penalty terms introduced to limit deviations beyond predefined thresholds. The study evaluates the impact of varying these penalty magnitudes on learning speed, convergence behavior, and tracking quality. Simulations are conducted in MATLAB/Simulink, demonstrating that the appropriate selection and application of penalties improve training efficiency without compromising control performance.