ROME: Memorization Insights from Text, Logits and Representation

Previous studies on model memorization have been limited by their reliance on comparing outputs with training corpora. particularly for models like LLaMA with closed pre-training datasets. This paper introduces a novel approach that avoids direct access to training data, named ROME (ROME  refers to the four letters in “memorization”, it also indicates “Rome (memorization) was not built in a day”.). Instead, it focuses on datasets where text chunks express fixed semantics, categorized into three types: context-independent, conventional, and factual. We redefine memorization as the ability to produce correct answers within these categories. Our analysis explores the contrasts in behavior patterns between memorized and non-memorized samples, focusing on differences in logits and representations of generated texts. Experimental results demonstrate that models consistently exhibit higher confidence when producing memorized answers.