This paper addresses the indoor positioning problem with the objective of enhancing localization accuracy using data from WiFi signals. The performance of the positioning system is highly sensitive to fluctuations in WiFi signal strength, which can negatively impact accuracy. To address this challenge, we propose a transformer-based algorithm, leveraging its ability to effectively model complex dependencies in data and mitigate the impact of signal fluctuations. Attention weights play a crucial role in enabling the transformer model to focus on relevant features; however, traditional softmax-based attention mechanisms often lack the flexibility needed to adapt to variable signal environments. By exploiting the unique characteristics of WiFi fingerprint data, we investigate Dirichlet distribution-based self-attention to replace the original mechanism for improved performance. Incorporating the Dirichlet distribution allows for greater control over attention scores, enabling the model to assign more robust, context-dependent weights to each token. This customization dynamically adjusts the attention focus on various signal features, even amidst fluctuating WiFi signals, thereby enhancing the model’s adaptability and overall positioning accuracy. Experimental results illustrate the efficiency of the proposed methodology, which outperforms existing approaches and achieves an accuracy improvement of up to 64%.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

NormSoftmax Attention: Improving Transformer Model Performance

  • Phuong Anh Nguyen,
  • Anh Ngoc Le

摘要

This paper addresses the indoor positioning problem with the objective of enhancing localization accuracy using data from WiFi signals. The performance of the positioning system is highly sensitive to fluctuations in WiFi signal strength, which can negatively impact accuracy. To address this challenge, we propose a transformer-based algorithm, leveraging its ability to effectively model complex dependencies in data and mitigate the impact of signal fluctuations. Attention weights play a crucial role in enabling the transformer model to focus on relevant features; however, traditional softmax-based attention mechanisms often lack the flexibility needed to adapt to variable signal environments. By exploiting the unique characteristics of WiFi fingerprint data, we investigate Dirichlet distribution-based self-attention to replace the original mechanism for improved performance. Incorporating the Dirichlet distribution allows for greater control over attention scores, enabling the model to assign more robust, context-dependent weights to each token. This customization dynamically adjusts the attention focus on various signal features, even amidst fluctuating WiFi signals, thereby enhancing the model’s adaptability and overall positioning accuracy. Experimental results illustrate the efficiency of the proposed methodology, which outperforms existing approaches and achieves an accuracy improvement of up to 64%.