A soft prompt learning method for medical text classification with simulated human cognitive capabilities
摘要
Medical text classification aims to identify the category to which a short medical text belongs. Recent research indicates that the “prompt-based learning” paradigm, especially the soft prompt, can improve the performance in text classification tasks by bridging the gap between pre-training objectives and downstream tasks. However, current research on soft prompt learning neglects the relation between the pseudo tokens and raw sentences when generating embeddings for the template. This study investigates, for the first time, how to simulate human cognitive processes in medical text classification tasks using soft prompt learning based on attention mechanisms. The proposed approach pays more attention to the parts of the raw sentence that are more relevant to the category label when generating embeddings for the pseudo tokens, resembling the reasoning process humans go through during text classification. Experiments conducted on two datasets, KUAKE-QIC and CHIP-CTC, indicate that the F1-macro scores of the proposed approach are 0.8064 and 0.8434, which outperform the benchmark and previous prompt learning approaches. In addition, the corresponding experiments also demonstrate the generalization ability and few-shot learning capability of the proposed method.