Analyzing the LMS Weight Error Covariance Matrix: An Exact Expectation Approach

One of the most relevant tasks of high-dimensional estimation is the computation of the parameters covariance matrix. This matrix offers valuable insights into the uncertainties associated with the estimation process. In the context of adaptive filtering, the weight error covariance matrix is a key parameter that determines how the filter adapts to input statistics and noise levels. Additional statistical information about the weight error vector allows one to depict a more precise description of the stochastic coupling between the adaptive weights. This paper concentrates on an in-depth study of the least mean squares asymptotic weight error covariance matrix, employing the exact expectation analysis. Through this examination, a key conclusion emerges: The symmetries commonly engendered by traditional analyses are not present in the actual covariance matrix. Instead, these symmetries are artifacts of the widespread assumption of independence. In summary, the advanced analysis performed in this work reveals a significantly more nuanced learning behavior exhibited by the least mean squares algorithm, challenging the conventional understanding put forth by traditional approaches. The theoretical predictions are confirmed by extensive simulations.