Human teleoperators are often required to assume the perspective and control of remote robot(s). In other words, they have to embody an avatar. The growing need to alternate control between more than one avatar can compromise embodiment, typically measured in terms of one’s sense of body ownership, agency, and change in body scheme. This study questions: how many avatars can a single operator effectively embody? Here, participants controlled 2, 3, and 4 avatars in a virtual reality (VR) environment and were evaluated for task performance (i.e., task completion times), motion sickness (i.e., Fast Motion Sickness Scale; FMS), and their subjective sense of embodiment and corresponding subdimensions (Virtual Embodiment Questionnaire; VEQ). We found significantly lower agency scores for controlling 3 avatars relative to controlling 2 avatars. Scores in ownership and change in body scheme scores were unaffected by manipulating the number of controlled avatars. It is worth noting that task performance and motion sickness levels were also unaffected by increases in the number of controlled avatars. Our findings reveal that there may be hidden costs to increasing the number of controlled avatars, namely in a sense of agency. We discuss the implications of this.