<p>Measuring academic performance through citation counts and derived indicators has become a sacred ritual in contemporary research evaluations. In this well-established liturgy, Google Scholar (GS) has become a widely used tool among researchers due to its accessibility and broad coverage. While the Bibliometrics community has consistently criticized GS for its lack of editorial control and opaque indexing criteria, its indicators are used to assess research impact in particular disciplines. This practice may lead to questionable citation practices, which only become evident when compared with more curated databases, such as Web of Science (Wos). This letter theoretically introduces and empirically tests an easy-to-implement indicator (the Citation Discrepancy Index, CDI) designed to explore the often-ignored extreme discrepancies in author citation counts across bibliographic databases and scientific fields. While some discrepancy is expected due to differences in coverage and publishing practices, this indicator might assist research evaluation committees in identifying cases where the divergence exceeds a reasonable or acceptable threshold, potentially signalling inconsistencies that merit closer scrutiny. Importantly, high CDI scores should be interpreted with caution: they are diagnostic signals of potential citation inconsistencies, not evidence of misconduct. To provide empirical support for the proposed indicator, several field-sensitive discrepancy thresholds are built to detect GS profiles requiring a full qualitative review.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

When citation counts don’t count: detecting author citation discrepancies across bibliographic databases to improve research evaluations

  • Manuel Goyanes,
  • Enrique Orduña-Malea

摘要

Measuring academic performance through citation counts and derived indicators has become a sacred ritual in contemporary research evaluations. In this well-established liturgy, Google Scholar (GS) has become a widely used tool among researchers due to its accessibility and broad coverage. While the Bibliometrics community has consistently criticized GS for its lack of editorial control and opaque indexing criteria, its indicators are used to assess research impact in particular disciplines. This practice may lead to questionable citation practices, which only become evident when compared with more curated databases, such as Web of Science (Wos). This letter theoretically introduces and empirically tests an easy-to-implement indicator (the Citation Discrepancy Index, CDI) designed to explore the often-ignored extreme discrepancies in author citation counts across bibliographic databases and scientific fields. While some discrepancy is expected due to differences in coverage and publishing practices, this indicator might assist research evaluation committees in identifying cases where the divergence exceeds a reasonable or acceptable threshold, potentially signalling inconsistencies that merit closer scrutiny. Importantly, high CDI scores should be interpreted with caution: they are diagnostic signals of potential citation inconsistencies, not evidence of misconduct. To provide empirical support for the proposed indicator, several field-sensitive discrepancy thresholds are built to detect GS profiles requiring a full qualitative review.