Conventional ultrasonic testing has limitations on the plug-in fillet weld due to the particularity of the inserted structure in pressure vessel. Therefore, a new method of flexible phased array technique is presented in this paper, which places the probe on the inner wall of the tube during detection. Firstly, beam focusing model for the water-steel interface of flexible phased array is established, and the calculation formula of the delay time is derived. Then, the flexible phased array probe parameters for testing the plug-in fillet weld were designed using CIVA software, including sound field simulation and defect response. Finally, one plug-in fillet weld sample was manufactured and tested. Experiment results indicated that this technique can effectively detect six types of common defect such as the lack of fusion, incomplete penetration, groove crack, porosity, slag, tube fusion surface crack, and is expected to achieve engineering application.