This report systematically introduces the process and outcomes of participating in the AI CUP 2023 Privacy Protection and Electronic Medical Record Standardization competition. We employed various innovative methods, particularly opting for the use of the “EleutherAI/pythia-410m-deduped” large language model in the final stages. In terms of related work, the report cites previous research on GPT-3’s role in summarizing medical literature, emphasizing the potential advantages and limitations of large language models in handling medical documents. Regarding algorithmic approaches and model architecture, the report delves into their text generation model, including its design, configuration, and training methods. Notably, the team conducts a comparative analysis of different models (70m, 410m, and 1b), examining their scores during training and at different stages. We extensively discuss innovative training methods, data processing, and batch processing, highlighting their efforts in model training, prediction, and post-processing. Lastly, the report provides a detailed analysis of the competition data, including the quantity and performance comparison of each category in the test set. In conclusion, the report discusses model performance, training periods, and comparisons between different models, pointing out potential directions for future improvements. The comprehensive and in-depth nature of the report underscores the team’s efforts and innovations in the competition.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

A Deep Dive into the Application of Pythia for Enhancing Medical Information De-identification in the AI CUP 2023

  • Zhi-En Li,
  • Hong-Yang Zheng,
  • Kuan-Chieh Mao,
  • Zhi-Wen Wei

摘要

This report systematically introduces the process and outcomes of participating in the AI CUP 2023 Privacy Protection and Electronic Medical Record Standardization competition. We employed various innovative methods, particularly opting for the use of the “EleutherAI/pythia-410m-deduped” large language model in the final stages. In terms of related work, the report cites previous research on GPT-3’s role in summarizing medical literature, emphasizing the potential advantages and limitations of large language models in handling medical documents. Regarding algorithmic approaches and model architecture, the report delves into their text generation model, including its design, configuration, and training methods. Notably, the team conducts a comparative analysis of different models (70m, 410m, and 1b), examining their scores during training and at different stages. We extensively discuss innovative training methods, data processing, and batch processing, highlighting their efforts in model training, prediction, and post-processing. Lastly, the report provides a detailed analysis of the competition data, including the quantity and performance comparison of each category in the test set. In conclusion, the report discusses model performance, training periods, and comparisons between different models, pointing out potential directions for future improvements. The comprehensive and in-depth nature of the report underscores the team’s efforts and innovations in the competition.