Automatic Verbalizer for Extracting Fine-Grained Customer Opinions from Non-English Social Media Comments

Social media comments serve as useful references for reflecting customer preference and help to devise appropriate business strategies. Apart from knowing the overall preference of a product, obtaining fine-grained opinions in specific aspects can be more insightful. Masked language modeling (MLM) is one of effective approaches to extract aspect opinions. It extracts fine-grained information by utilizing a Large Language Model, such as BERT, to make opinion-bearing textual predictions. In practice, the predicted results are not always as expected due to the nature of training data and computational constraints for MLM fine-tuning. A verbalizer is needed to map unexpected predictions to desired outcomes. Existing literature mainly focuses on handling English comments with single-token MLM predictions. This study addresses concerns for non-English multiple-token predictions and proposes a practical methodology to develop an automatic verbalizer using variants of the k-nearest neighbors (KNN) classifier. The proposed approach does not rely on a predefined dictionary. It predicts an aspect opinion using contextualized embedding vectors which encompass the information from the opinion-bearing textual prediction together with its underlying context. Concerns for handling multiple-token predictions are discussed and workable solutions are proposed. Variants of KNN classifier are employed to consolidate information obtained from multiple tokens. The performance of proposed verbalizers is examined on a large Chinese Meituan-Dianping dataset with over 40k genuine user reviews. Methods for MLM task decomposition, prompt construction and model finetuning are also addressed in this study.