<p>With the exponential growth of scientific literature in STEM (Science, Technology, Engineering, and Mathematics) fields, researchers face the dual challenge of information overload and the increasing burden of peer review. Traditional bibliometric metrics often fail to reflect intrinsic research value, while manual review is constrained by scalability. To address these issues, this paper proposes CICQ, a unified framework that integrates Citation Impact and Content Quality for automated literature evaluation. A core contribution of this framework is the development of an automated peer review mechanism, which leverages fine-tuned Large Language Models (LLMs) to perform deep semantic analysis of manuscript content (evaluating Contribution, soundness, and presentation). By unifying this intrinsic content assessment with extrinsic citation sentiment analysis, the CICQ framework effectively prevents the underestimation of high-quality or high-potential papers (Sleeping Beauties) that may lack immediate citation attention. Furthermore, to overcome the challenge of defining ground truth in literature quality, we construct an interval-censored dataset based on publication venue tiers and employ an ordered logistic regression model. This approach effectively quantifies the contribution of each dimension, providing a comprehensive, interpretable, and automated solution for evaluating scientific contribution in the AI era.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Cicq: a unified framework integrating citation impact and content quality for automated literature evaluation

  • Xi Chen,
  • Xuyang Meng,
  • Shijiong Lv,
  • Ya Zhang,
  • Yang Chen

摘要

With the exponential growth of scientific literature in STEM (Science, Technology, Engineering, and Mathematics) fields, researchers face the dual challenge of information overload and the increasing burden of peer review. Traditional bibliometric metrics often fail to reflect intrinsic research value, while manual review is constrained by scalability. To address these issues, this paper proposes CICQ, a unified framework that integrates Citation Impact and Content Quality for automated literature evaluation. A core contribution of this framework is the development of an automated peer review mechanism, which leverages fine-tuned Large Language Models (LLMs) to perform deep semantic analysis of manuscript content (evaluating Contribution, soundness, and presentation). By unifying this intrinsic content assessment with extrinsic citation sentiment analysis, the CICQ framework effectively prevents the underestimation of high-quality or high-potential papers (Sleeping Beauties) that may lack immediate citation attention. Furthermore, to overcome the challenge of defining ground truth in literature quality, we construct an interval-censored dataset based on publication venue tiers and employ an ordered logistic regression model. This approach effectively quantifies the contribution of each dimension, providing a comprehensive, interpretable, and automated solution for evaluating scientific contribution in the AI era.