Prompt Engineering for Safety Red-Teaming: A Case Study on Text-to-Image Diffusion Models
摘要
Diffusion models for text-to-image (T2I) synthesis have demonstrated exceptional capabilities for generating high-quality content. However, this progress has raised notable concerns about potential misuse, particularly in creating copyrighted, prohibited, and restricted content, or NSFW (not safe for work) images. While efforts have been made to mitigate such problems, either by implementing a safety filter at the evaluation stage or by fine-tuning models to eliminate undesirable concepts or styles, the effectiveness of these safety measures in dealing with a wide range of prompts remains largely unexplored. In this work, we introduce two primary prompt engineering approaches, Prompting4Debugging (P4D) (Chin et al., Prompting4debugging: Red-teaming text-to-image diffusion models by finding problematic prompts. International Conference on Machine Learning (2024)) and Ring-A-Bell (Tsai et al., Ring-a-bell! how reliable are concept removal methods for diffusion models? International Conference on Learning Representations (2024)), acting as debugging and red-teaming tools to automatically find problematic prompts for diffusion models to test the reliability of a deployed safety mechanism.