The approximate linear equation \(Ax\approx b\) with some columns of A error-free can be solved via mixed least squares-total least squares (MTLS) model by minimizing a nonlinear function. This paper is devoted to the Gauss–Newton iteration for the MTLS problem. With an appropriately chosen initial vector, each iteration step of the standard Gauss–Newton method requires to solve a smaller-size least squares problem, in which the QR of the coefficient matrix needs a rank-one modification. To improve the convergence, we devise a relaxed Gauss–Newton (RGN) method by introducing a relaxation factor and provide the convergence results as well. The convergence is shown to be closely related to the ratio of the square of subspace-restricted singular values of [A, b]. The RGN can also be modified to solve the total least squares (TLS) problem. Applying the RGN method to a Bursa–Wolf model in parameter estimation, numerical results show that the RGN-based MTLS method behaves much better than the RGN-based TLS method. Theoretical convergence properties of the RGN-MTLS algorithm are also illustrated by numerical tests.
