Machine learning is deployed in various clinical and healthcare informatics applications, with centralized and decentralized learning schemes, each offering performance and security advantages and disadvantages. In centralized machine learning, all the data travels to a central location where the machine learning training code runs on it. A central server potentially represents a single point of failure - which is one of the issues of centralized learning. Another issue is the need for all participants to trust the central authority with their datasets. In a decentralized machine learning solution, data stays at the participating local sites, while the machine learning training code travels to each site. It needs parties to run a common binary code on each of their datasets and trust the incoming program, thus avoiding a single point of failure, but potentially creating a security hazard with malicious code. This can be addressed by using a mutually agreed upon signed binary code. Another issue is the training run time in decentralized learning due to multiple hops between different dataset locations. System designers often face tradeoffs between the higher performance of centralized machine learning vs. the better security of decentralized machine learning. In this work, we propose a novel Collaborative Federated Learning (CFL) solution that combines the advantages of centralized and decentralized machine learning schemes, without compromising security. We executed our simulation using synthetic data for 30 iterations to observe the behavior of code and dataset sizes with various incremental data sharing options. Interestingly, we observed that when more data is shared centrally, data security issues become more pervasive, however, machine learning training performance improves proportionally. The challenge is to balance between performance and security considerations by partitioning appropriate amount of data to be shared centrally.