Precise dynamic models or fine tuning is necessary for traditional dynamical system controller. However, in many scenarios, precise dynamic models may be hard acquire or they just change by time. And aimless tuning doesn’t always yield ideal results. In this paper, we propose a model-free online controller based on reinforcement learning. We apply Q-learning algorithm and update new control value according to control values and real-time system feedbacks based on RLS iteratively in a model-free method. The controller is validated in gym simulation environment and compared with LQR controller. Results proved that our controller performs as well as LQR and can automatically adapt to model changes.