Scholarlink

<p>A sequence of three randomized controlled trials (RCTs) is conducted to support a small online business’ decision of whether and how to implement AI in the creation of email marketing content. Recent developments in frequentist statistical decision theory are used to accommodate small samples available for testing in the small-business setting. The RCTs comprise three test policy cells with email content created by (i) salaried writers (“human”), (ii) a large-language model (“LLM”), and (iii) a “hybrid” combination of a human editing the content created by the LLM, respectively. When a “no email” control policy is included, all three test cells approximately double the gross profits from orders relative to the control cell. The RCTs vary whether the hybrid cell is edited by a salaried writer or the marketing team. The LLM cells vary whether the AI is pre-trained using historic emails or uses a prompt-based generative pre-trained transformer (“GPT”). Decision theory always selects one of the AI cells over the standard human policy on the basis of total annual profit net of related labor and software overhead. </p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Large Language Models and Creative Content Design: a case study of email marketing at Wine Access

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年1月13日

查看全文
  • Jean-Pierre Dubé,
  • Ariel Xu

摘要

A sequence of three randomized controlled trials (RCTs) is conducted to support a small online business’ decision of whether and how to implement AI in the creation of email marketing content. Recent developments in frequentist statistical decision theory are used to accommodate small samples available for testing in the small-business setting. The RCTs comprise three test policy cells with email content created by (i) salaried writers (“human”), (ii) a large-language model (“LLM”), and (iii) a “hybrid” combination of a human editing the content created by the LLM, respectively. When a “no email” control policy is included, all three test cells approximately double the gross profits from orders relative to the control cell. The RCTs vary whether the hybrid cell is edited by a salaried writer or the marketing team. The LLM cells vary whether the AI is pre-trained using historic emails or uses a prompt-based generative pre-trained transformer (“GPT”). Decision theory always selects one of the AI cells over the standard human policy on the basis of total annual profit net of related labor and software overhead.