Accurate joint force/torque measurements are required for many applications, which can be obtained by multiplying the drive signals with the joint drive gains. However, the joint drive gains are usually unknown or inaccurate. In the previous work, an overdetermined linear system is constructed when the robot tracks one reference trajectory without a payload and another with a calibrated payload fixed on the robot, and the drive gains are estimated based on the ordinary/weighted/total least squares solutions. However, the physical feasibility conditions of payload dynamic parameters have not been considered, and the identification results may be affected by outliers. This paper proposes a robust identification method to identify all drive gains simultaneously. First, an inverse dynamic model of the payload is derived when the robot tracks the same trajectory both without a payload and with a payload fixed on the robot. The robot dynamic parameters are not incorporated into this model. Based on the payload inverse dynamic model, a novel iteratively reweighted least squares (IRLS)-based filter is introduced to filter out outliers caused by vibration and unmodeled dynamics. Then, joint drive gains are estimated by solving a least squares problem with physical feasibility constraints. The filtered results of drive signals are used instead of the measured values to avoid distorting the observation matrix. The effectiveness of the proposed method is experimentally validated on a 6-DOF cooperative robot (Chinrobo CRB-7).