Scholarlink

Objective <p>This study aims to develop an enhanced Transformer model for predicting mild cognitive impairment (MCI) using data from the China Health and Retirement Longitudinal Study (CHARLS), focusing on handling mixed data types and improving predictive accuracy.</p> Methods <p>The Transformer integrates categorical (integer-encoded) and continuous (floating-point) data, using multi-head attention with four heads to capture complex relationships. Preprocessing involved separate embedding layers for categorical data and feed-forward networks for continuous data. The model was compared with SVM and XGBoost, trained for 150 epochs with RMSProp and a cosine annealing scheduler. Key metrics included accuracy, Mean Absolute Error (MAE) tolerance, and training loss. An attention heatmap was generated to visualize feature importance.</p> Results <p>The Transformer outperformed SVM and XGBoost, achieving over 90% accuracy at an MAE tolerance of 3.5. The model showed rapid convergence, with loss stabilizing within 20 epochs. The attention heatmap highlighted key features, confirming the effectiveness of the multi-head attention mechanism in identifying relevant variables.</p> Conclusion <p>The enhanced Transformer model offers superior accuracy and efficiency in predicting cognitive decline compared to traditional models. Its capacity to process both continuous and categorical data and its interpretability through attention mechanisms make it a promising tool for early detection of neurodegenerative diseases, potentially improving clinical decision-making and interventions.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Leveraging transformer models to predict cognitive impairment: accuracy, efficiency, and interpretability

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2025年2月7日

查看全文
  • Kai Ma,
  • Junzhi Zhang,
  • Xinhang Huang,
  • Mengyang Wang

摘要

Objective

This study aims to develop an enhanced Transformer model for predicting mild cognitive impairment (MCI) using data from the China Health and Retirement Longitudinal Study (CHARLS), focusing on handling mixed data types and improving predictive accuracy.

Methods

The Transformer integrates categorical (integer-encoded) and continuous (floating-point) data, using multi-head attention with four heads to capture complex relationships. Preprocessing involved separate embedding layers for categorical data and feed-forward networks for continuous data. The model was compared with SVM and XGBoost, trained for 150 epochs with RMSProp and a cosine annealing scheduler. Key metrics included accuracy, Mean Absolute Error (MAE) tolerance, and training loss. An attention heatmap was generated to visualize feature importance.

Results

The Transformer outperformed SVM and XGBoost, achieving over 90% accuracy at an MAE tolerance of 3.5. The model showed rapid convergence, with loss stabilizing within 20 epochs. The attention heatmap highlighted key features, confirming the effectiveness of the multi-head attention mechanism in identifying relevant variables.

Conclusion

The enhanced Transformer model offers superior accuracy and efficiency in predicting cognitive decline compared to traditional models. Its capacity to process both continuous and categorical data and its interpretability through attention mechanisms make it a promising tool for early detection of neurodegenerative diseases, potentially improving clinical decision-making and interventions.