Exploring the application of open peer review in academic evaluation: an analysis of H1 Connect recommended papers
摘要
Analyzing the open peer review is instrumental in delving into the value of papers from an expert perspective, thereby providing valuable references for applying open review data in research evaluation practice. Based on the H1 Connect platform, this study analyzed open peer review data from papers in Neoplasms, Cardiovascular Diseases, and Respiratory Tract Diseases. From both numerical and textual perspectives, we analyzed the attention and recognition of papers, sentiment characteristics and scientific research contributions identified by large language models, then conducted regression analysis between these contributions and impact indicators. The findings revealed that open peer review indicators exhibit significant topic-specific variations and were positively correlated with impact indicators. Sentiment in open peer reviews is generally neutral, with positive sentiments tending to be focused on results and discussion of papers. Biomedical contributions predominantly focused on knowledge advancement and clinical application, with few studies addressing economic and social benefits. Most papers made contributions in one specific area, while multi-type contributions remained uncommon. Papers with contributions generally achieved higher academic recognition and social attention, particularly those featuring Clinical Trial Outcomes and Public Health Policy. Academic evaluation can leverage open peer reviews to understand the contributions of papers deeply, considering contribution diversity and disciplinary differences.