Leveraging onboard flight data to improve dynamics modeling accuracy and further enhance control performance has attracted considerable attention in recent years. However, challenges including noisy measurements and phase delays within estimation are involved during the implementation of this idea. To this end, a Gaussian process (GP)-based method for online learning residual dynamics is proposed in this study, and the contributions are threefold. Firstly, the fitting discrepancy of the GP model is augmented as a new state to be estimated, and an extended Kalman filter (EKF) is developed to estimate the mean and variance of the augmented states for GP learning. Secondly, the derivative information of the posterior GP is utilized to automatically adjust the bandwidth of the EKF to match the non-stationary evolution of dynamics residuals. Finally, a fixed-lag Kalman smoother is developed to overcome the phase delay issue of EKF estimation. The effectiveness and advantages of the proposed method are demonstrated in the last section through numerical experiments of wing rock dynamics learning.