错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Real-time speech-to-speech translation with urgency preservation in time-sensitive scenarios

  • Simone Pinna,
  • Francesca Maridina Malloci,
  • Mirko Marras,
  • Diego Reforgiato Recupero,
  • Daniele Riboni,
  • Giuseppe Scarpi

摘要

In a global society where collaboration increasingly involves speakers of different languages, communication barriers remain a significant challenge. The problem becomes particularly relevant in urgent scenarios, where critical information must be delivered rapidly and understood correctly across speakers of several languages. To address this challenging issue, we introduce a multilingual framework for near real-time speech-to-speech translation that explicitly detects and preserves urgency. The system extends the cascade paradigm by integrating audio- and text-based urgency recognition with urgency-aware speech synthesis, and has been implemented as an open-source prototype accessible through WebRTC. As an additional contribution, we created the first publicly available multilingual dataset explicitly annotated for urgency, comprising 200 recordings in English, French, German, Dutch, and Italian. The framework has been evaluated through both quantitative and qualitative analyses. The experimental results indicate that our system achieves real-time performance while maintaining translation accuracy and effectively preserving perceived urgency. While most current solutions are offered by commercial providers and deployed on the cloud, raising significant privacy concerns, our system is released open-source and can be executed on premises using a mid-range server. The dataset is publicly available at https://zenodo.org/records/17502378.