Scholarlink

<p>Generative AI systems are increasingly deployed in digital mental health contexts, where personalization is essential for engagement yet simultaneously amplifies ethical risks. Hyper-personalized companion chatbots have been associated with dependency and blurred relational boundaries, whereas under-personalized clinical chatbots often struggle to sustain engagement among young adults facing rising loneliness and unmet mental health needs. This tension gives rise to what we term the personalization paradox: in safety-critical emotional AI, personalization is both necessary and potentially risky. To explore this challenge, we propose Boundary-Aware Therapeutic Personalization (BTP), an ethically grounded reframing of personalization that emphasizes contextual inquiry, strategic questioning, reflective summarization, and explicit boundary maintenance rather than affective mimicry or simulated intimacy. We examined the feasibility and behavioral implications of this framework through a three-phase empirical evaluation. Phase 0 assessed whether therapeutic engagement was possible without identity storage, indicating moderate-to-high continuance intention (M = 6.9/10) and a strong association between personalization and engagement (<i>r</i> = .85). Phase 1 conducted a comparative evaluation of four mental health chatbots (Replika, Wysa, Youper, Dr.CareSam) across seven therapeutic competencies, integrating assessments from two chain-of-thought LLM evaluators (ChatGPT-4.0, Claude 3.5 Sonnet) and clinical expert review (licensed psychologist, 30 + years’ experience; Cronbach’s α = 0.837). Question-led systems were rated as more professionally appropriate and safer than affective-mirroring designs. Phase 2 examined boundary-aware enhancements across severity-graded scenarios, providing illustrative evidence of how BTP principles may translate into practice. Rather than aiming for maximal human-likeness or intimacy simulation, BTP adopts a deliberately minimal, boundary-preserving approach to personalization. Taken together, these findings suggest that engagement and safety may not necessarily be mutually exclusive: when grounded in boundary-aware design, therapeutic AI systems can support ethically aligned personalization while reducing risks related to dependency and boundary erosion.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Resolving the personalization paradox in therapeutic AI: a boundary-aware framework for ethical personalization

  • 期刊论文

  • 发表日期：

    2026年2月8日

查看全文
  • Boyoung Kang

摘要

Generative AI systems are increasingly deployed in digital mental health contexts, where personalization is essential for engagement yet simultaneously amplifies ethical risks. Hyper-personalized companion chatbots have been associated with dependency and blurred relational boundaries, whereas under-personalized clinical chatbots often struggle to sustain engagement among young adults facing rising loneliness and unmet mental health needs. This tension gives rise to what we term the personalization paradox: in safety-critical emotional AI, personalization is both necessary and potentially risky. To explore this challenge, we propose Boundary-Aware Therapeutic Personalization (BTP), an ethically grounded reframing of personalization that emphasizes contextual inquiry, strategic questioning, reflective summarization, and explicit boundary maintenance rather than affective mimicry or simulated intimacy. We examined the feasibility and behavioral implications of this framework through a three-phase empirical evaluation. Phase 0 assessed whether therapeutic engagement was possible without identity storage, indicating moderate-to-high continuance intention (M = 6.9/10) and a strong association between personalization and engagement (r = .85). Phase 1 conducted a comparative evaluation of four mental health chatbots (Replika, Wysa, Youper, Dr.CareSam) across seven therapeutic competencies, integrating assessments from two chain-of-thought LLM evaluators (ChatGPT-4.0, Claude 3.5 Sonnet) and clinical expert review (licensed psychologist, 30 + years’ experience; Cronbach’s α = 0.837). Question-led systems were rated as more professionally appropriate and safer than affective-mirroring designs. Phase 2 examined boundary-aware enhancements across severity-graded scenarios, providing illustrative evidence of how BTP principles may translate into practice. Rather than aiming for maximal human-likeness or intimacy simulation, BTP adopts a deliberately minimal, boundary-preserving approach to personalization. Taken together, these findings suggest that engagement and safety may not necessarily be mutually exclusive: when grounded in boundary-aware design, therapeutic AI systems can support ethically aligned personalization while reducing risks related to dependency and boundary erosion.