Scholarlink

<p>The proliferation of scientific podcasts has generated an extensive repository of educational content, rich in specialized terminology, diverse topics, and expert dialogues. Here, we introduce a computational framework designed to enhance large language models by leveraging this informational content from publicly accessible audio podcasts across science, technology, engineering, mathematics, and medicine (STEMM). This dataset, comprising over 3700 hours of audio content, was transcribed to generate over 42 million text tokens. Our model, PodGPT, integrates this wealth of complex dialogue found in audio podcasts to improve understanding of natural language nuances, cultural contexts, as well as scientific and medical knowledge. PodGPT also employs retrieval augmented generation (RAG) on a vector database, providing real-time access to emerging scientific literature. Evaluated on multiple benchmarks, PodGPT demonstrated an average improvement of 1.82 percentage points over standard open-source benchmarks and 2.43 percentage points when augmented with evidence from the RAG pipeline. Moreover, it showcased an average improvement of 1.18 percentage points in its zero-shot multilingual transfer ability, effectively generalizing to different linguistic contexts. By harnessing the untapped potential of podcast content, PodGPT advances natural language processing and conversational AI, offering enhanced capabilities for STEMM research and education.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

PodGPT: an audio-augmented large language model for research and education

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2025年7月7日

查看全文
  • Shuyue Jia,
  • Subhrangshu Bit,
  • Edward Searls,
  • Meagan V. Lauber,
  • Pengrui Fan,
  • William M. Wang,
  • Lindsey A. Claus,
  • Varuna H. Jasodanand,
  • Divya Veerapaneni,
  • Rhoda Au,
  • Vijaya B. Kolachalama

摘要

The proliferation of scientific podcasts has generated an extensive repository of educational content, rich in specialized terminology, diverse topics, and expert dialogues. Here, we introduce a computational framework designed to enhance large language models by leveraging this informational content from publicly accessible audio podcasts across science, technology, engineering, mathematics, and medicine (STEMM). This dataset, comprising over 3700 hours of audio content, was transcribed to generate over 42 million text tokens. Our model, PodGPT, integrates this wealth of complex dialogue found in audio podcasts to improve understanding of natural language nuances, cultural contexts, as well as scientific and medical knowledge. PodGPT also employs retrieval augmented generation (RAG) on a vector database, providing real-time access to emerging scientific literature. Evaluated on multiple benchmarks, PodGPT demonstrated an average improvement of 1.82 percentage points over standard open-source benchmarks and 2.43 percentage points when augmented with evidence from the RAG pipeline. Moreover, it showcased an average improvement of 1.18 percentage points in its zero-shot multilingual transfer ability, effectively generalizing to different linguistic contexts. By harnessing the untapped potential of podcast content, PodGPT advances natural language processing and conversational AI, offering enhanced capabilities for STEMM research and education.