Federated Learning (FL) enables collaborative model training across multiple clients without sharing raw data, ensuring privacy in sensitive domains like healthcare. Traditional FL often relies on a central server for model aggregation from different clients, leading to significant limitations and risks such as a single-point failure, communication bottlenecks, scalability issues, privacy risks, and limited resilience, especially when working with heterogeneous, multimodal data across clients. To overcome these limitations, a Decentralized Collaborative Multi-Client Federated Learning (DCM-FL) approach is proposed to eliminate its dependency on a central server, enabling direct collaboration between clients. The DCM-FL setup consists of three independent clients using different architectures, such as CNN (Client1), VGG16 (Client2), and ResNet50 (Client3), each containing the CBIS-DDSM dataset with various distributions to predict breast cancer from breast mammogram images. The architectural dependency among clients during aggregation is mitigated by integrating two different aggregation methods, such as layer-type aggregation (Method 1) and knowledge distillation (Method 2). The results show that Method 1 gradually improves the accuracy of Client1 from 51.53% to 68.40% by Round 3, whereas Method 2 achieves an accuracy of 67.18% in Round 1 and sustains further. The trade-off between computation and communication highlights that Method 1 is efficient for homogeneous model architectures with similar layer types for direct weight sharing, and Method 2 is better suited for heterogeneous environments where direct weight sharing is not feasible, but requires higher computation and communication costs.