Towards Weight-Space Interpretation of Low-Rank Adapters for Diffusion Models
摘要
Low-rank adapters (LoRAs) have emerged as an efficient method for customizing large-scale diffusion models, but their internal representations remain poorly understood. We present a comprehensive investigation of the interpretability of adapter weight-spaces for image diffusion models. To that end, we open-source a dataset of 100,000 Stable Diffusion adapters fine-tuned across a hierarchy of image concepts amounting to 264 leaf classes, complete with training metadata. Through systematic analysis, we demonstrate that adapter weights encode meaningful semantic information about their training data, enabling direct interpretation without image generation. We evaluate multiple weight-space representations, including raw parameters, statistical summaries, and learned embeddings, to determine their effectiveness in predicting training data characteristics. To demonstrate real-world impact, we apply our findings to the critical task of detecting potentially harmful content on newly introduced NSFW (Not Safe For Work) toy dataset of Stable Diffusion LoRAs fine-tuned on harmful content. This work advances the interpretability of adapter-based fine-tuning and provides practical tools for understanding and auditing adapted diffusion models.