Word Sense Induction in Russian: Evaluation of Corpora Preprocessing Techniques and Model Selection

The chapter proposes a solution to the problem of word sense induction (WSI) in Russian. WSI refers to the task of resolving ambiguity by deriving features from corpora without using knowledge bases, dictionaries, and predefined lists of word senses. This study examines the impact of data preprocessing on the performance of WSI techniques using clustering algorithms for context vector representations based on a series of BERT models for the Russian language. Various methods of data preprocessing were analyzed, including tokenization, lemmatization, stop words’ removal, and selection of pre-trained models of context vector representations. Experiments have shown that the presence or absence of lemmatization does not affect clustering results, while other factors such as the removal of stop words and the choice of the data vectorization model can significantly affect clustering. The results of the study can be applied to the procedures of semantic annotation of text corpora.