Does Bert Know How ‘Virus’ Evolved: Tracking Usage Changes in Chinese Textual Data

Recent studies indicated a trend of quantifying lexical semantic changes with distributional models. In this study, we investigated whether state-of-the-art language models can tell us the story of how a word developed its senses over time. Specifically, we exploited the Bert model to obtain sense representations and quantitatively track usage changes after performing sense classification for each occurrence of targets in a historical newspaper dataset(People’s Daily(1954–2003). Our experiment provided a positive answer to the research question, as the model has an overall precision score of 91.82% on classifying senses against human judgments. We also charted usage changes of targets, which demonstrates a possible way to (semi-)automatically observe the development of word meanings.