Multi-innovation-based online variable projection algorithm for a class of nonlinear models

Separable nonlinear models (SNMs) serve as potent instruments for system identification, data analysis, and machine learning. However, the online identification of SNMs poses a greater challenge compared to their offline counterparts, primarily due to the dynamic nature of nonlinear systems. Traditional approaches, including the recursive Gauss–Newton and recursive variable projection methods, falter in managing parameter interdependence, culminating in sluggish convergence and suboptimal outcomes. Addressing these limitations, our study introduces a pioneering multi-innovation-based recursive variabl projection (MIRVP) algorithm, which extends the RVP algorithm with a multi-innovation strategy. This strategy enables the algorithm to handle the interaction of the linear and nonlinear parameters more effectively, by using multiple past innovations instead of only the current one, thus improving the identification effect. The proposed algorithm’s effectiveness has been validated through tests on synthetic data, real-life industrial control tracking scenarios such as the Box–Jenkins gas furnace data and Glass tube drawing processing, as well as on the training of large-scale neural networks. The results demonstrate that our algorithm outperforms existing methods in terms of convergence speed and robustness