<p>Several matrix factorization algorithms are employed in machine learning applications. Among these, Non-negative Matrix Factorization (NMF) gained attention due to the ability to extract meaningful features from inherently non-negative data, such as documents, images or videos. However, such data are often privacy-sensitive, which necessitates formal privacy guarantees of the machine learning model training algorithm. Additionally, modern data are typically stored in different nodes or clients, rather than a centralized server. Conventional decentralized privacy-preserving schemes suffer from too much noise and consequently, much lower utility compared to their centralized counterparts. This motivates us to develop an efficient privacy-preserving NMF algorithm that can operate on decentralized data, and can closely approximate the performance of centralized/non-privacy-preserving approach, while offering strict privacy guarantees. We design our method and demonstrate our results in such a way that the clients/data holders have the control to select the degree of privacy guarantee based on the required utility. We show the effectiveness of our proposed algorithm on six real datasets. Our experimental results show that our proposed method easily outperforms conventional privacy-preserving scheme, while achieving close approximation of the non-privacy-preserving approach under some parameter choices.</p>