Cybernetic Sentinels: Unveiling the Impact of Safety Data Selection on Model Security in Supervised Fine-Tuning

Large Language Models (LLMs) continue to advance in capabilities, yet this progress comes with an expanding array of safety risks. This paper discusses concerns related to models that prioritize being helpful over ensuring safety in the process of instruction-tuning. We investigate the comparative effects of randomly selecting guidance data versus clustering safety guidance data during fine-tuning on LLaMA2-7B. Results indicate that the latter can more effectively mitigate security risks of open-source models with minimal performance impact. Additionally, we examine the impact of various response combinations on model security performance. Experimental results show that concise refusal responses moderately enhance model security defense capabilities, albeit at the potential expense of response quality in regular question-answering tasks. Finally, this selected 2000 fine-tuning data for model security training is available for researchers’ use, which can be found at https://github.com/wangruihui0429/Security-through-Strategic-Data-Selection .