Implementation of the Thematic Text Analysis Algorithm Using Machine Learning

In this paper, the goal is to develop an algorithm for thematic text analysis based on machine learning methods. Thematic analysis is the extraction of keywords and phrases from the text and the compilation of the top of n-grams by relevance. To solve this problem, we used the pre-trained BERT NLP model from SberDevices, which was trained mainly on the Russian data set. This fact is extremely important, since the project is primarily aimed at working with Russian texts. This approach was chosen after analyzing the most famous modern linguistic systems and not all of them are able to work correctly with the Russian language. As the result of the work there is an algorithm that can provide substantial results for the set of goals, provided that the subject of the processed texts is sufficiently represented in the training data set. In the future, it is planned to use this algorithm for another study in the field of psychometrics and social network analysis, which requires the processing of a large number of texts.