Randomized neural network models, such as the random vector functional link and extreme learning machine (ELM), offer several advantageous characteristics when compared to conventional backpropagation-based neural network models. However, traditional learning algorithms for these randomized neural network models face certain obstacles. The presence of outlier training samples and weight noise can significantly impact the performance of the trained neural network. To overcome these challenges, this study presents a comprehensive approach that effectively addresses both weight noise and outlier sample issues simultaneously. The ELM model is used as an example to demonstrate the proposed approach. For each training sample, we develop an error term that includes the impact of weight noise. It is important to note that the developed noise-tolerant error term differs from the common fitting error that only considers fitting accuracy. Therefore, the conventional method of constructing a robust training algorithm cannot be used. In this paper, we propose using the sum of rooted noise-tolerant error terms as the training objective. However, the proposed objective function is not convex. To address this, we generalize the iteratively reweighted least squares (IRLS) methodology, which is originally designed to handle the standard case, to develop our robust noise-tolerance algorithm. The convergence properties of the proposed algorithm are theoretically discussed. Simulation results demonstrate that the proposed algorithm outperforms the state-of-the-art robust algorithm for randomized neural network models.