Scholarlink

<p>Framing analysis, an extensively used, multi-disciplinary social science research method, requires substantial manpower and time to code and uncover human-level understanding of story contexts. However, recent advances in deep learning have led to a qualitative jump in algorithm-assisted methods, with large language models (LLMs) like BERT and GPT going beyond surface characteristics to infer the semantic properties of a text. In this study, we explore the application of the BERT&#xa0;for&#xa0;natural language inference (NLI), which leverages bidirectional context and rich embeddings to assist scholars in identifying contextual information in media texts for quantitative framing analysis. More specifically, we investigate the capability of LLMs to identify generic media frames by comparing the results from a zero-shot analysis using BERT-NLI to those from human analysis. We find that the reliability of detecting generic frames varies significantly across different datasets, indicating that even a large LLM like BERT-NLI, trained on millions of texts from diverse sources, cannot be uniformly trusted across different contexts. Nonetheless, LLMs might be employed productively in specific contexts after careful consideration of their agreement with human-generated ratings.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Towards algorithmic framing analysis: expanding the scope by using LLMs

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2025年3月13日

查看全文
  • Xianwen Kuang,
  • Jun Liu,
  • Haiyang Zhang,
  • Simon Schweighofer

摘要

Framing analysis, an extensively used, multi-disciplinary social science research method, requires substantial manpower and time to code and uncover human-level understanding of story contexts. However, recent advances in deep learning have led to a qualitative jump in algorithm-assisted methods, with large language models (LLMs) like BERT and GPT going beyond surface characteristics to infer the semantic properties of a text. In this study, we explore the application of the BERT for natural language inference (NLI), which leverages bidirectional context and rich embeddings to assist scholars in identifying contextual information in media texts for quantitative framing analysis. More specifically, we investigate the capability of LLMs to identify generic media frames by comparing the results from a zero-shot analysis using BERT-NLI to those from human analysis. We find that the reliability of detecting generic frames varies significantly across different datasets, indicating that even a large LLM like BERT-NLI, trained on millions of texts from diverse sources, cannot be uniformly trusted across different contexts. Nonetheless, LLMs might be employed productively in specific contexts after careful consideration of their agreement with human-generated ratings.