Noise propagation in linear stationary iterations

We consider a stationary iteration for solving a linear system of arbitrary order. The method includes, e.g. Kaczmarz iteration, the Landweber iteration and the SOR (Gauss-Seidel) iteration. A study of the behavior of the iterates, both theoretically and experimentally, is performed. In particular we compare the behavior with and without noise in the data. The results give insight into the interplay between noise free and noisy iterates. For comparision we also included a Krylov type method CGLS in the experiments. As expected CGLS works well for noise free data but also tends to amplify the noise faster than the other methods, thus making it more critical when to stop the iterations.