<p>Medical text classification aims to identify the category to which a short medical text belongs. Recent research indicates that the “prompt-based learning” paradigm, especially the soft prompt, can improve the performance in text classification tasks by bridging the gap between pre-training objectives and downstream tasks. However, current research on soft prompt learning neglects the relation between the pseudo tokens and raw sentences when generating embeddings for the template. This study investigates, for the first time, how to simulate human cognitive processes in medical text classification tasks using soft prompt learning based on attention mechanisms. The proposed approach pays more attention to the parts of the raw sentence that are more relevant to the category label when generating embeddings for the pseudo tokens, resembling the reasoning process humans go through during text classification. Experiments conducted on two datasets, KUAKE-QIC and CHIP-CTC, indicate that the F1-macro scores of the proposed approach are 0.8064 and 0.8434, which outperform the benchmark and previous prompt learning approaches. In addition, the corresponding experiments also demonstrate the generalization ability and few-shot learning capability of the proposed method.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

A soft prompt learning method for medical text classification with simulated human cognitive capabilities

  • Yu Wang,
  • Luyao Zhou,
  • Weimin Zhang,
  • Feifan Zhang,
  • Yuan Wang

摘要

Medical text classification aims to identify the category to which a short medical text belongs. Recent research indicates that the “prompt-based learning” paradigm, especially the soft prompt, can improve the performance in text classification tasks by bridging the gap between pre-training objectives and downstream tasks. However, current research on soft prompt learning neglects the relation between the pseudo tokens and raw sentences when generating embeddings for the template. This study investigates, for the first time, how to simulate human cognitive processes in medical text classification tasks using soft prompt learning based on attention mechanisms. The proposed approach pays more attention to the parts of the raw sentence that are more relevant to the category label when generating embeddings for the pseudo tokens, resembling the reasoning process humans go through during text classification. Experiments conducted on two datasets, KUAKE-QIC and CHIP-CTC, indicate that the F1-macro scores of the proposed approach are 0.8064 and 0.8434, which outperform the benchmark and previous prompt learning approaches. In addition, the corresponding experiments also demonstrate the generalization ability and few-shot learning capability of the proposed method.