<p>In rater-mediated assessment, annotating refers to a process in which raters note, comment, and/or mark on scripts while assessing. While annotations are studied as written corrective feedback that enhances learners’ performance, their impact on raters’ assessment processes and outcomes remains unclear. We conducted a quasi-experiment to explore how annotating activity would affect peer raters’ cognitive functioning and psychometric qualities of peer ratings in assessment of English-to-Chinese interpreting. Quantitative and qualitative data analyses show that: a) annotation in interpreting assessment was a content-based, error-focused, and potentially efficiency-driven activity; b) although annotation seemed to involve more mental effort, it facilitated the peer raters’ comprehension, memorization, and metacognition during assessing interpretation; and c) overall, annotation contributed to higher levels of generalizability, (concurrent) validity, and accuracy of the peer ratings across the rating dimensions. Practical and pedagogical implications of these findings for peer assessment are discussed.</p>