Personalized federated multilingual machine translation with multi-level knowledge fusion
摘要
Training multilingual machine translation models using federated learning aims to enhance performance across different translation directions. However, existing federated learning approaches for multilingual machine translation face challenges in addressing the heterogeneity of multi-client data. In this paper, we propose a personalized federated multilingual machine translation method with multi-level knowledge fusion, designed for realistic multilingual neural machine translation while preserving data privacy.Our method presents an innovative global–local aggregation technique that uses Bayesian fusion to integrate the global knowledge gathered from all clients with the client-specific local knowledge. This strategy ensures optimal results for each client’s unique translation direction. Additionally, we adopt an iterative clustering strategy based on gradient similarity to dynamically assign feature-similar clients to appropriate aggregation centers. This approach maximizes the learning of both language-specific and generic knowledge within and across clusters, thereby reducing parameter interference among clients.We assess our approach using both many-to-one and many-to-many datasets. Experimental results, including ablation studies, demonstrate the effectiveness of the proposed approach. Furthermore, poisoning attack experiments confirm the robustness of the method.