Scholarlink

We propose FLARE, a novel few-shot unlearning framework for LLMs. FLARE conceptualizes the unlearning tasks as relearning tasks, focusing on learning the new labels for the unwanted instances. FLARE integrates task-specific and instance-specific information into a shared source prompt called TI prompt and adapts TI prompt to downstream unlearning tasks. We conduct the experiments across different few-shot unlearning tasks, and the results demonstrate that FLARE significantly outperforms state-of-the-art baselines despite fine-tuning only 1% to 2% of the LLM’s parameters.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Few-Shot Unlearning for Large Language Models Through Pretrained Soft Prompts

  • 会议论文

  • 发表日期：

    2025年

查看全文
  • Zee Hen Tang,
  • Mi-Yen Yeh

摘要

We propose FLARE, a novel few-shot unlearning framework for LLMs. FLARE conceptualizes the unlearning tasks as relearning tasks, focusing on learning the new labels for the unwanted instances. FLARE integrates task-specific and instance-specific information into a shared source prompt called TI prompt and adapts TI prompt to downstream unlearning tasks. We conduct the experiments across different few-shot unlearning tasks, and the results demonstrate that FLARE significantly outperforms state-of-the-art baselines despite fine-tuning only 1% to 2% of the LLM’s parameters.