The exponential growth of data and model complexity has rendered traditional centralized recommender system approaches insufficient, necessitating the adoption of distributed learning paradigms. This article provides a comprehensive survey of distributed learning algorithms, focusing on their fundamental principles, architectural variations, and critical applications within recommender systems. We delineate the distinctions between centralized, distributed, and decentralized systems, highlighting how distributed approaches address challenges such as computational bottlenecks, inherently distributed datasets, and evolving privacy concerns. The survey explores key distributed learning paradigms, including parallel, federated, and decentralized learning, demonstrating their progressive evolution from optimizing computational speed to ensuring data privacy, robustness, and sovereignty. Specifically, we delve into the application of these algorithms in recommender systems, examining techniques like distributed matrix factorization for scalability, federated learning for privacy-preserving personalization, and parallel distributed training for complex hybrid models. The report details how these methods enable recommender systems to handle massive datasets, provide real-time updates, and safeguard sensitive user information. Finally, we discuss the ongoing challenges and promising future directions in this dynamic field, emphasizing the continuous pursuit of enhanced communication efficiency, robustness to data heterogeneity, advanced privacy mechanisms, and decentralized trust for building more resilient and ethically sound systems.