Polarized images have significant application value in optical research and computer vision. Collaborative training of polarized images can effectively integrate multi-source data and enhance the generalization ability and performance of models. However, due to the fact that these data are usually distributed across multiple devices and have high privacy sensitivity, such as distributed devices or terminals equipped with polarized cameras, direct central training may face the risk of data leakage and insufficient privacy protection. To address this issue, this paper proposes a polarization image enhancement method based on federated learning to effectively safeguard data privacy. To optimize the model aggregation in the federated learning framework, a loss-weighted aggregation strategy is designed, combining multi-source information from each client to construct a global model for distributed low-light image enhancement. Experiments conducted on two real-world datasets demonstrate that the proposed method not only significantly reduces training time but also outperforms traditional centralized methods in terms of privacy protection. To ensure reproducibility, the source code and demonstration videos can be found at https://pan.baidu.com/s/1dzWMZgnGYl92uyabAkfhXQ (Extraction code: zs6h).