<p>The interrater agreement of ratings on a nominal scale regarding a group of targets (individuals, objects, etc.) is usually computed by kappa-type indices. Recently, a homogeneity index for a qualitative variable was considered to define a single target measure of agreement between raters for each target, and to propose a global measure of agreement for the whole group of targets allowing to overcome limitations affecting traditional kappa-type indices. In this paper, the sampling properties and the asymptotic distribution of the proposed index are investigated. Finally, a simulation study is performed to demonstrate the accuracy of the index and an application to real educational data is provided.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Statistical inference for indices of interrater agreement based on a homogeneity index

  • Giuseppe Bove,
  • Daniela Marella

摘要

The interrater agreement of ratings on a nominal scale regarding a group of targets (individuals, objects, etc.) is usually computed by kappa-type indices. Recently, a homogeneity index for a qualitative variable was considered to define a single target measure of agreement between raters for each target, and to propose a global measure of agreement for the whole group of targets allowing to overcome limitations affecting traditional kappa-type indices. In this paper, the sampling properties and the asymptotic distribution of the proposed index are investigated. Finally, a simulation study is performed to demonstrate the accuracy of the index and an application to real educational data is provided.