Scholarlink

With increased embedding of artificial intelligence (AI) technologies in organizational processes, the effectiveness of human–AI collaboration is an important topic for research. It is, however, challenging to quantify the performance and value of such collaboration using any level of methodological rigor. This article analyzes and critically examines the existing methodologies used to assess the efficacy of human–AI interaction within organizational contexts. It categorizes prevailing measurement methods as quantitative (e.g., KPIs, ROI, productivity), qualitative (e.g., user surveys, interviews, case studies), and mixed-methods models (e.g., TAM, UTAUT, adaptive collaboration models). Drawing on this background, the paper presents a new evaluation framework that combines technical, organizational, and human factors of collaboration in a multidimensional matrix. The paper’s discussion section addresses the practical relevance of such methodologies, highlighting their advantages and shortcomings and how organizational context determines the choices among methodologies. The paper concludes by specifying directions for future research as it formulates more integrated, context-sensitive measures of human–AI collaboration.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Methods for Measuring the Effectiveness of Human-AI Collaboration in an Organization

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Radosław Wolniak,
  • Agnieszka Kowalska-Styczeń,
  • Izabela Jonek-Kowalska,
  • Aneta Michalak

摘要

With increased embedding of artificial intelligence (AI) technologies in organizational processes, the effectiveness of human–AI collaboration is an important topic for research. It is, however, challenging to quantify the performance and value of such collaboration using any level of methodological rigor. This article analyzes and critically examines the existing methodologies used to assess the efficacy of human–AI interaction within organizational contexts. It categorizes prevailing measurement methods as quantitative (e.g., KPIs, ROI, productivity), qualitative (e.g., user surveys, interviews, case studies), and mixed-methods models (e.g., TAM, UTAUT, adaptive collaboration models). Drawing on this background, the paper presents a new evaluation framework that combines technical, organizational, and human factors of collaboration in a multidimensional matrix. The paper’s discussion section addresses the practical relevance of such methodologies, highlighting their advantages and shortcomings and how organizational context determines the choices among methodologies. The paper concludes by specifying directions for future research as it formulates more integrated, context-sensitive measures of human–AI collaboration.