Refining the Ethical Guidelines for Conversational AI by Distinguishing Human Communication from Human–AI Interaction

The growing integration of conversational AI (CAI) into users’ emotional lives raises pressing ethical concerns. This article examines the 2024 suicide of an American teenager, Sewell, following prolonged interaction with an AI chatbot, highlighting possible blind spots in existing CAI ethical guidelines. While existing guidelines address transparency and harm reduction, challenges arise when users experience interaction as if it were genuine communication, forming deep emotional attachments. To understand such challenges, this paper contrasts interpersonal communication with human–CAI interaction, emphasizing asymmetries in intentional structure and contextual embeddedness. Whereas human communication is oriented toward mutual understanding and rooted in personal and social life, CAI lacks true intentionality and contextual awareness, even as it simulates communicative cues, thus inviting emotional projection. Drawing on this distinction, the article proposes three normative considerations for refining current CAI guidelines: clarifying interaction through design, improving sensitivity to user context, and framing CAI as a complement rather than substitute for human connection. These reflections may contribute to ongoing efforts in developing CAI systems in an ethically appropriate way.