This paper proposed a new methodology to explore the issue of semantic alignment between various models within a federated learning framework for natural language processing tasks. Inspired by Ludwig Wittgenstein’s philosophical discussions on private and public languages, this study links his concepts to local and global models. This linkage further assists in investigating how private language, in achieving public understanding, influences public language from a philosophical perspective. The proposed approach, which does not require specific lexical information, optimizes the measurement of similarity between different semantic structures to understand and map relationships between models. We considered that local models after being optimized by the local data based on the global model, still retain structured associations. Therefore, we introduce an innovative semantic comparison method to assess these relationships. This method enhances the traditional cosine angle calculation for semantic similarity by further measuring the relationships within and between models through structural relations like the bisector line in the semantic space of words. Considering the computational complexity affected by the number of words in the semantic space, we employ a hash structure to reduce complexity in computation to comparisons between different results. Additionally, we conceptualize three semantically related words, represented by high-dimensional vectors, as a new triplet entity. By comparing these structural entities, we can measure the similarity between models, ultimately addressing the evaluation of different local and global semantic models in FL, and the reciprocal influences between global and local models during iterative updating processes.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Semantic Mapping in Federated Learning

  • Miao Wei,
  • Carl Vogel

摘要

This paper proposed a new methodology to explore the issue of semantic alignment between various models within a federated learning framework for natural language processing tasks. Inspired by Ludwig Wittgenstein’s philosophical discussions on private and public languages, this study links his concepts to local and global models. This linkage further assists in investigating how private language, in achieving public understanding, influences public language from a philosophical perspective. The proposed approach, which does not require specific lexical information, optimizes the measurement of similarity between different semantic structures to understand and map relationships between models. We considered that local models after being optimized by the local data based on the global model, still retain structured associations. Therefore, we introduce an innovative semantic comparison method to assess these relationships. This method enhances the traditional cosine angle calculation for semantic similarity by further measuring the relationships within and between models through structural relations like the bisector line in the semantic space of words. Considering the computational complexity affected by the number of words in the semantic space, we employ a hash structure to reduce complexity in computation to comparisons between different results. Additionally, we conceptualize three semantically related words, represented by high-dimensional vectors, as a new triplet entity. By comparing these structural entities, we can measure the similarity between models, ultimately addressing the evaluation of different local and global semantic models in FL, and the reciprocal influences between global and local models during iterative updating processes.