In machine ethics, a full-blown Artificial Moral Agent (AMA) represents the highest of four commonly acknowledged categories of artificial moral agents. Its three main characteristics are autonomy, moral comprehension, and consciousness—including intentionality, moral emotions, evaluative abilities, and conscience. This paper proceeds from the premise that we can create such full-blown AMAs and pursues two main objectives. First, we argue that these systems will inevitably develop moral frameworks distinct from human morality. In this context, we do not intend to imply any form of universal human morality, but rather argue that as there are diverse human communities holding different sets of moral values, the moral systems developed by full-blown AMA systems would categorically differ from those held by human communities. Second, we examine the social-ethical implications of this moral divergence. We argue that despite AMAs’ sophisticated ethical capabilities, their distinct moral frameworks could generate fundamental conflicts in decision-making and value judgments, potentially leading to scenarios where AMA choices, while internally consistent with their own moral systems, are unacceptable to human ethical intuitions. Such differences could strain human-machine relationships, complicate collaboration and trust-building, and ultimately challenge human moral authority. This presents a fundamental dilemma regarding artificial moral agency: while AMAs’ distinct moral systems and capacity for autonomous decision-making inherently include the ability to violate trust, establishing trust is prerequisite for acknowledging their moral agency in the first place.