Evaluating Media Knowledge Capabilities of Intelligent Search Dialogue Systems: A Case Study of ChatGPT and New Bing

This study establishes an index system for evaluating the media knowledge capabilities of intelligent search dialogue systems (ISDS), which includes three primary indicators: dialogue ability, special ability, and output value. Survey data from relevant users are used to analyze the reliability and validity of the index system. An empirical comparison indicates that ChatGPT outperforms New Bing in understanding and dialogue capabilities, whereas New Bing excels in output content value. This paper also proposes new dimensions for ISDS, emphasizing their positive impact on knowledge production and dissemination. Nonetheless, ISDS also pose a series of risks and problems, including data privacy leaks and ethical challenges, which urgently need researching and addressing.