An integrated indicator for evaluating scientific papers: considering academic impact and novelty

The assessment of scientific papers has long been a challenging issue. Although numerous studies have proposed quantitative indicators for assessing scientific papers, these studies overlooked the citation characteristics and the novelty of scientific knowledge implied in the textual information of papers. Therefore, this paper constructs an integrated indicator to evaluate scientific papers from both citation and semantic perspectives. Firstly, we propose weighted citations to measure the academic impact of scientific papers, which takes time heterogeneity and citation sentiment factors into consideration. Secondly, we capture the novelty of scientific papers from a semantic perspective, utilizing FastText to represent papers as text embeddings and applying the local outlier factor to calculate it. To validate the performance of our approach, the bullwhip effect domain and the ACL Anthology corpus are used for case studies. The results demonstrate that our indicator can effectively identify outstanding papers, thus providing a more comprehensive evaluation method for evaluating academic research.