The surge in social media discussions has increased the need for automated summarization to save time and manage content overload. However, existing summarization datasets are not suitable for this purpose, as they either do not cover discussions, multiple modalities, or both. To end this, we present MMRSUM, a comprehensive multimodal discussion summarization dataset boasting 7,130 meticulously curated threads. Given the diversity of languages in India, with more than 1 billion people, Hinglish is crucial for facilitating communication across linguistic groups, reflecting cultural integration, and enhancing professional and educational environments. Additionally, we present HMMRSUM, an innovative multimodal codemixed discussion summarization dataset with 7,130 threads in Hinglish. Each thread within these datasets encompasses a solicitation post enriched with both textual and visual elements, facilitating a holistic understanding. Subsequent comments further augment comprehension by integrating textual and visual components. To refine the summarization process, we propose an innovative approach employing Relevant-Cluster-based Multi-stage Summarization (RCMS). Our model demonstrates strong performance on both English and codemixed multimodal datasets. It not only surpasses existing benchmarks by a notable margin of \(\approx 6\%\) but also sets a new standard for future research endeavors. The detailed dataset samples are available at https://github.com/M-Groot7/MMRSUM-HMMRSUM.git

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

From Conversations to Insights: A Multimodal Approach to Discussion Summarization

  • Punit Kumar Singh,
  • Nishant Kumar,
  • Hrushik Mehta,
  • Sriparna Saha

摘要

The surge in social media discussions has increased the need for automated summarization to save time and manage content overload. However, existing summarization datasets are not suitable for this purpose, as they either do not cover discussions, multiple modalities, or both. To end this, we present MMRSUM, a comprehensive multimodal discussion summarization dataset boasting 7,130 meticulously curated threads. Given the diversity of languages in India, with more than 1 billion people, Hinglish is crucial for facilitating communication across linguistic groups, reflecting cultural integration, and enhancing professional and educational environments. Additionally, we present HMMRSUM, an innovative multimodal codemixed discussion summarization dataset with 7,130 threads in Hinglish. Each thread within these datasets encompasses a solicitation post enriched with both textual and visual elements, facilitating a holistic understanding. Subsequent comments further augment comprehension by integrating textual and visual components. To refine the summarization process, we propose an innovative approach employing Relevant-Cluster-based Multi-stage Summarization (RCMS). Our model demonstrates strong performance on both English and codemixed multimodal datasets. It not only surpasses existing benchmarks by a notable margin of \(\approx 6\%\) but also sets a new standard for future research endeavors. The detailed dataset samples are available at https://github.com/M-Groot7/MMRSUM-HMMRSUM.git