The PI parameters design is very challenging in engineering and a small variation of PI values can lead to significant differences in both simulation and experiments results. Traditional trial and error method is very cumbersome in designing PI parameters, while the frequency domain methods require designers to be familiar with accurate model expressions and frequency performances judgement. Deep reinforcement learning (DRL) method is model-free, which can obtain an ideal PI parameters automatically through machine learning. A specific DRL PI parameters regulation example with power amplifier is fully demonstrated in this article, in which the control target needs to be tracked is sinusoidal waveform. The framework and its implementation are shown in this paper and the results are compared by simulation, which verifies the simplicity and effectiveness of the DRL method.