Scholarlink

Background <p>Understanding the alignment and contributions of research to the United Nations Sustainable Development Goals (UN SDGs) is essential for guiding global progress toward these critical targets. Several SDG mapping approaches have been proposed and tested by organisations and researchers but have not produced consistent results. With its capacity to analyse vast datasets and identify patterns, AI-powered search functionality has been presented as an innovative mechanism for tracking, analysing and reporting SDG research to assess progress towards targets and facilitate evidence-based decisions. This study aimed to assess the reliability of automated mapping approaches utilised by online research databases in mapping published evidence syntheses to the UN SDG-3, Good Health &amp; Wellbeing.</p> Methods <p>This study mapped systematic and scoping reviews published in <i>JBI Evidence Synthesis</i> to SDG- 3, Good Health and Wellbeing. Four unique raters independently assessed 204 evidence syntheses based on relevance to SDG-3. These four raters included AI in three established databases and a manual ‘human’ assessment. Inter-rater reliability was assessed using Light’s Kappa.</p> Results <p>Concurrence occurred for 52% of publications. Inter-rater reliability indicated ‘minimal agreement’ among the four raters in mapping the 204 evidence syntheses to SDG-3. Discrepancies in the publications mapped to SDG-3 across the four raters may be explained by the different taxonomies used by the databases, different machine learning algorithms, and constantly evolving search strategies. Our results indicate significant room for improvement in achieving greater consensus among approaches, including AI algorithms designed to identify such publications.</p> Conclusion <p>The findings of this study point to the need for SDG mapping tools that are practical and effective, as the 2030 agenda deadline nears and progress on numerous targets lags. Identifying evidence across the global ecosystem is critical, but AI’s reliability to contribute to this has yet to be established with confidence. It signals to the scientific community, policymakers, funders and others the importance of critically reflecting on how research is captured and aligned to the SDGs.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Cracking the code: AI’s role in mapping evidence syntheses to the United Nations sustainable development goals

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2025年11月25日

查看全文
  • Bianca Pilla,
  • Jennifer Stone,
  • Zoe Jordan

摘要

Background

Understanding the alignment and contributions of research to the United Nations Sustainable Development Goals (UN SDGs) is essential for guiding global progress toward these critical targets. Several SDG mapping approaches have been proposed and tested by organisations and researchers but have not produced consistent results. With its capacity to analyse vast datasets and identify patterns, AI-powered search functionality has been presented as an innovative mechanism for tracking, analysing and reporting SDG research to assess progress towards targets and facilitate evidence-based decisions. This study aimed to assess the reliability of automated mapping approaches utilised by online research databases in mapping published evidence syntheses to the UN SDG-3, Good Health & Wellbeing.

Methods

This study mapped systematic and scoping reviews published in JBI Evidence Synthesis to SDG- 3, Good Health and Wellbeing. Four unique raters independently assessed 204 evidence syntheses based on relevance to SDG-3. These four raters included AI in three established databases and a manual ‘human’ assessment. Inter-rater reliability was assessed using Light’s Kappa.

Results

Concurrence occurred for 52% of publications. Inter-rater reliability indicated ‘minimal agreement’ among the four raters in mapping the 204 evidence syntheses to SDG-3. Discrepancies in the publications mapped to SDG-3 across the four raters may be explained by the different taxonomies used by the databases, different machine learning algorithms, and constantly evolving search strategies. Our results indicate significant room for improvement in achieving greater consensus among approaches, including AI algorithms designed to identify such publications.

Conclusion

The findings of this study point to the need for SDG mapping tools that are practical and effective, as the 2030 agenda deadline nears and progress on numerous targets lags. Identifying evidence across the global ecosystem is critical, but AI’s reliability to contribute to this has yet to be established with confidence. It signals to the scientific community, policymakers, funders and others the importance of critically reflecting on how research is captured and aligned to the SDGs.