In the new media era, the strengthening of netizens’ discourse power constitutes a more complex public opinion field. Phenomena such as information overload, rumor spreading, public opinion reversal and group polarization are flooding the network, which may subtly affect people's value orientation and behavior judgment. College students are the future and hope of the country, which is of great significance to personal growth and the future development of the country. Based on the traditional TF-IDF algorithm, this paper increases the weight of specific part of speech and word length, which can better distinguish news events described by different news documents. In view of the dynamic and temporal nature of network news reports, this paper divides the hot topic detection task into multiple time segments in chronological order, and obtains the final hot topic through two steps of initial detection and topic merging. This topic is based on the Hadoop platform of network public opinion hot spot detection, in the data collection, preprocessing and hot spot detection (clustering analysis) and other parts of the use of HDFS and MapRedcue parallel computing framework and other big data processing technology, effectively solve the traditional technology to deal with the lack of massive text data.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Research on Hotspot Detection of Online Public Opinion Based on the Hadoop Platform

  • Hui Wang

摘要

In the new media era, the strengthening of netizens’ discourse power constitutes a more complex public opinion field. Phenomena such as information overload, rumor spreading, public opinion reversal and group polarization are flooding the network, which may subtly affect people's value orientation and behavior judgment. College students are the future and hope of the country, which is of great significance to personal growth and the future development of the country. Based on the traditional TF-IDF algorithm, this paper increases the weight of specific part of speech and word length, which can better distinguish news events described by different news documents. In view of the dynamic and temporal nature of network news reports, this paper divides the hot topic detection task into multiple time segments in chronological order, and obtains the final hot topic through two steps of initial detection and topic merging. This topic is based on the Hadoop platform of network public opinion hot spot detection, in the data collection, preprocessing and hot spot detection (clustering analysis) and other parts of the use of HDFS and MapRedcue parallel computing framework and other big data processing technology, effectively solve the traditional technology to deal with the lack of massive text data.