Neural Network Language Models, Mutual Sense Affinity, and Short Text Ranking According to Closeness to a Semantic Standard

The selection of papers related to a given topic involves not only an analysis of the relevance of the vocabulary of each paper to the topic of interest of the end user, but also taking the ultimate goal of the user himself into account (i.e., for which tasks the selection is made). The main requirement here can be represented as the sorting of information sources by the degree of reflection of the most significant concepts of a given topical area at maximum compactness and nonredundancy of narration. In the ideal case, we obtain an estimation for the mutual semantic dependence of texts relative to the most rational (i.e., standard) variants of description for knowledge fragments represented by them. In the current study, the problem of implementation of such ranking of texts is solved on the basis of analysis of their mutual semantic affinity by application of neural network language models of the Transformer architecture. By applying these models, we map sentences of the text into multi-dimensional vectors called embeddings. Here, the estimation of semantic affinity (i.e., the “strength” of the semantic relationship) of analyzed text fragments can be defined by the measure of closeness of vectors corresponding them. The resulting rating of a paper associated by the authors with the closeness of its abstract to the semantic standard is defined by the product of estimation of the “strength” of the semantic relationship of the analyzed paper with the rest of the collection and estimation of the connection in meaning for sentences in the abstract of the analyzed paper. The connection in meaning of an analyzed abstract assumes that its sentences must be related together by sense as much as possible. To form the optimal order of a user working with papers in the collection after its ranking, for each of the papers the search for the paper most close in sense to it is carried out using cosine similarity of corresponding embeddings. Here, the trajectory of the navigation of the user’s collection is built “from top to bottom” from a publication with a higher rating to a publication with a lower rating that is closest to it in meaning.