Finding variables and statistical metrics to describe rank distributions of lexemes is a relevant linguistic task. We analyze the validity of lingvo-statistical parameters (the rank mean and entropy) for describing frequency dictionary of fiction. The comparative use of the Weibull and Haustein functions as approximating ones for the values of the parameters in question is also investigated. The research draws on a representative sample from the Corpus of the Russian Short Stories (1900–1930) (total volume is more than 1,000,000 tokens). The rank mean is shown to be only a relative valid parameter for describing a large-scale corpus of fiction, while the relative validity of entropy is greatly affected by the nature of the texts analyzed. The Weibull function is proved to be the preferable one for the approximation of the parameters’ growth.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Validity of Lingvo-Statistical Parameters for the Corpus of Fiction

  • Alexander Grebennikov,
  • Mikhail Koryshev,
  • Ekaterina Ivanova,
  • Tatiana Skrebtsova

摘要

Finding variables and statistical metrics to describe rank distributions of lexemes is a relevant linguistic task. We analyze the validity of lingvo-statistical parameters (the rank mean and entropy) for describing frequency dictionary of fiction. The comparative use of the Weibull and Haustein functions as approximating ones for the values of the parameters in question is also investigated. The research draws on a representative sample from the Corpus of the Russian Short Stories (1900–1930) (total volume is more than 1,000,000 tokens). The rank mean is shown to be only a relative valid parameter for describing a large-scale corpus of fiction, while the relative validity of entropy is greatly affected by the nature of the texts analyzed. The Weibull function is proved to be the preferable one for the approximation of the parameters’ growth.