Scholarlink

<p>Single-cell large language models (scLLMs) capture essential biological insights from vast single-cell atlases but struggle in out-of-context applications, where zero-shot predictions can be unreliable. To address this, here we introduce a single-cell parameter-efficient fine-tuning (scPEFT) framework that integrates learnable, low-dimensional adapters into scLLMs. By freezing the backbone model and updating only the adapter parameters, scPEFT efficiently adapts to specific tasks using limited custom data. This approach mitigates catastrophic forgetting, reduces parameter tuning by over 96% and decreases GPU memory usage by more than half, thus substantially enhancing the accessibility of scLLMs for resource-constrained researchers. When validated across diverse datasets, scPEFT outperformed zero-shot models and traditional fine-tuning in disease-specific, cross-species and undercharacterized cell population tasks. Its attention-mechanism analysis identified COVID-related genes associated with specific cell states and uncovered unique blood cell subpopulations, demonstrating the capacity of scPEFT for condition-specific interpretations. These findings position scPEFT as an efficient solution for enhancing the utility of scLLMs in general single-cell analyses.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Harnessing the power of single-cell large language models with parameter-efficient fine-tuning using scPEFT

  • 期刊论文

  • 发表日期：

    2025年12月31日

查看全文
  • Fei He,
  • Ruixin Fei,
  • Jordan E. Krull,
  • Yang Yu,
  • Xinyu Zhang,
  • Xianyu Wang,
  • Hao Cheng,
  • Mingyue Gao,
  • Li Su,
  • Yibo Chen,
  • Jinpu Li,
  • Baichuan Jin,
  • Yuzhou Chang,
  • Anjun Ma,
  • Qin Ma,
  • Dong Xu

摘要

Single-cell large language models (scLLMs) capture essential biological insights from vast single-cell atlases but struggle in out-of-context applications, where zero-shot predictions can be unreliable. To address this, here we introduce a single-cell parameter-efficient fine-tuning (scPEFT) framework that integrates learnable, low-dimensional adapters into scLLMs. By freezing the backbone model and updating only the adapter parameters, scPEFT efficiently adapts to specific tasks using limited custom data. This approach mitigates catastrophic forgetting, reduces parameter tuning by over 96% and decreases GPU memory usage by more than half, thus substantially enhancing the accessibility of scLLMs for resource-constrained researchers. When validated across diverse datasets, scPEFT outperformed zero-shot models and traditional fine-tuning in disease-specific, cross-species and undercharacterized cell population tasks. Its attention-mechanism analysis identified COVID-related genes associated with specific cell states and uncovered unique blood cell subpopulations, demonstrating the capacity of scPEFT for condition-specific interpretations. These findings position scPEFT as an efficient solution for enhancing the utility of scLLMs in general single-cell analyses.