Prompt-Based Learning on Large Protein Language Models Improves Signal Peptide Prediction

Signal peptides (SP) play a crucial role in protein localization in cells. The development of large protein language models (PLMs) provides a new opportunity for SP prediction. We applied a prompt-based learning framework, Parameter-Efficient Fine-Tuning (PEFT) for SP prediction, PEFT-SP, to effectively utilize pre-trained PLMs. We integrated low-rank adaptation (LoRA) into ESM-2 models to better leverage the protein sequence evolutionary knowledge of PLMs. Experiments show that PEFT-SP using LoRA enhances state-of-the-art results, leading to a maximum MCC gain of 0.372 for SPs with small training samples and an overall MCC gain of 0.048. Furthermore, we also employed two other prompt-based learning methods, i.e., Prompt Tuning and Adapter Tuning, into ESM-2 for SP prediction. More elaborate experiments show that PEFT-SP using Adapter Tuning can also improve the state-of-the-art results with up to 0.202 MCC gain for SPs with small training samples and an overall MCC gain of 0.030. LoRA requires fewer computing resources and less memory than the Adapter during the training stage, making it possible to adapt larger and more powerful protein models for SP prediction. The PEFT-SP framework is available at https://github.com/shuaizengMU/PEFT-SP . The web server for SP predic-tion leveraging the PEFT-SP framework is publicly available at https://www.mu-loc.org/peftsp/ .