Often, media producers exhibit certain types of bias that are passed through to news consumers, potentially shaping their opinions and perception of reality. This can have multiple negative effects, including polarized speech and general distrust on institutions and news producers themselves. The main goal of this research is to cover the European Portuguese news landscape and create an automated system that frames news pieces according to three axes: political bias, reliability, and objectivity. We adopt two methodologies: zero-shot prompting and a vector database technique, both working directly with a large language model and, more specifically, tested with EuroLLM 9B and LLaMA 3.3 70B. We report interesting results as the system reaches close to 90% accuracy on some cases. Furthermore, we observe that a medium-sized model works significantly better than a small-sized one.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Automated Media Assessment Using Large Language Models

  • João Carvalho,
  • Henrique Lopes Cardoso,
  • Rui Carreira

摘要

Often, media producers exhibit certain types of bias that are passed through to news consumers, potentially shaping their opinions and perception of reality. This can have multiple negative effects, including polarized speech and general distrust on institutions and news producers themselves. The main goal of this research is to cover the European Portuguese news landscape and create an automated system that frames news pieces according to three axes: political bias, reliability, and objectivity. We adopt two methodologies: zero-shot prompting and a vector database technique, both working directly with a large language model and, more specifically, tested with EuroLLM 9B and LLaMA 3.3 70B. We report interesting results as the system reaches close to 90% accuracy on some cases. Furthermore, we observe that a medium-sized model works significantly better than a small-sized one.