Content Moderator Startle Response: A Qualitative Study

Commercial content moderators review user-generated content (UGC) to ensure the posts meet platform policies, guidelines, community standards, and regional laws. While the majority of posted UGC is deemed acceptable, a large amount of content remains, which is classified as non-compliant and may include gore, violence, suicide, child sexual abuse material (CSAM), and pornography, to name a few. Because of this, content moderators have a greater prevalence of their nervous system activating a startle response, which can impact emotional, psychological, and physiological processes. Prior research on content moderators has failed to explore moderators’ initial reactions to content from the start of employment through tenure as the subjection to material and habituation increases. This study takes an in-depth look at moderators’ experiences from recruiting, through training, and production to better understand the content moderators’ startle response and factors that enable startle habituation. The current study sample consisted of 78 total respondents—38 content moderators in the Philippines employed by TaskUs Inc. and 40 in Estonia employed by Sutherland Global Services. Employee tenure ranged from 0 to 6 months. Succeeding our analysis, transparency, understanding, and preparedness were major themes identified as the critical factors found within both companies when exploring the activation of the startle response and facilitation of habituation following content exposure. These themes were prevalent in the employment life-cycle's recruiting, training, and production phases.