<p>Due to the inherent complexity of nonlinear problems, it is often difficult to effectively analyze and handle using traditional linear methods. Maximum likelihood estimation, as a statistical method for estimating model parameters, can effectively model and estimate complex nonlinear relationships. To expand the application of maximum likelihood estimation in nonlinear problems, this study focuses on the relative navigation of spacecraft at long-distance and establishes relevant kinematic and observation models. The relative navigation method for long-distance spacecraft is determined. Two robust iterative methods based on Gaussian-Newton method are improved using the maximum likelihood estimation method. The relevant evaluation indicators of the improved algorithm are compared with other algorithms under different noise conditions. The results indicated that under Gaussian observation noise conditions, the average normalized error square of the prediction results was reduced compared with the improved method. Under non-Gaussian observation noise conditions, the improved robust iterative Sigma point Kalman filter and unscented Kalman filter had average normalized error squared values of 0.885 and 2.512, respectively. Compared with adaptive filtering, the root mean square error of relative position prediction was reduced by 20.07% and 14.97%, and the root mean square error of relative velocity prediction was reduced by 26.31% and 20.84%, respectively. When the contamination probability was 0.2, the average root mean square errors of the two improved algorithms in predicting relative position and relative velocity were 2.53 × 10⁻², 2.86 × 10⁻², 5.613 × 10⁻⁴, and 6.187 × 10⁻⁴, respectively. The prediction accuracy of the research method under Gaussian noise conditions has improved compared with before the improvement, while its superiority is more evident under non-Gaussian conditions.</p>