Classifying informative tweets using feature enhanced pre-trained language model

Classifying tweets containing valuable information about COVID-19 is crucial for developing monitoring systems that provide the latest updates. Existing approaches for informative tweet classification considers only the last layer vector of a special token by ignoring the vectors of other tokens and the token vectors from the previous layers. The paper addresses this drawback by proposing a novel approach which (i) makes use of all the token vectors from the last four layers and (ii) leverages additional information in the form of POS tags and informative words. Experiment results show that the proposed approach outperforms all the existing approaches and achieves an accuracy of 92% and F1-score of 92.01% on the COVID-19 informative tweets dataset. The uniqueness of this paper is the attempt to leverage token vectors from the last four layers, additional information in the form of POS tags and informative words from COVID-19 informative tweets for classification.