Methods of User Opinion Data Crawling in Web 2.0 Social Network Discussions

It is widely accepted that nowadays a significant part of the content on the internet is generated by users of social media platforms which form the basis of Web 2.0. That is why modern media researchers use user-generated content to test their scientific hypotheses using automated data analysis methods and data mining tools. In this study we examine the main approaches to user opinion data crawling in modern social media platforms for subsequent analysis for scientific and research purposes. We propose a data collection approach based on reverse engineering of APK applications, which allows for data extraction from social networks that will not differ in completeness from data from mobile applications. A comparative analysis of the proposed methods in terms of completeness and execution speed is also carried out. According to our findings, implementing a custom REST API is the best approach as it is both reliable and computationally efficient.