Applying the Proposed Method for Creating Structural Models to Multilingual Collections of Text Documents Using Multi- and Monolingual BERT Models

This work is dedicated to the problem of automatically creating structural models of text documents written in different languages. We describe the proposed method for creating structural models which is based on extracting sentences that correspond to various aspects of documents using BERT Question Answering model. We analyze BERT-based models and consider their application to the task of question answering in different languages. The analysis shows that monolingual models can provide higher quality than multilingual ones. We consider the problem of creating structural models when processing multilingual collections of text documents. We analyze several approaches to this problem based on monolingual and multilingual Question Answering models and show that the most promising approach consists in preliminary classification of documents by languages and subsequent processing of each document using a respective monolingual model. We also demonstrate results of experiments carried out on sets of articles in English and Russian languages that prove the advantage of the proposed two-step approach.