Enhancing Word Sense Disambiguation Performance on WiC-TSV Dataset Using BERT-LSTM Model

WSD is a significant area of research in Natural Language Processing, as accurate disambiguation of word senses is crucial for many applications including text categorization, sentiment analysis, information retrieval, and machine translation. WiC (Word-in-Context) Target Sense Verification (TSV) dataset, a benchmark dataset for WSD is used. The paper proposes encoder-decoder models employing transformers and other deep learning architectures, to perform the job of word sense disambiguation, have been presented and examined in this study. BertTokenizer is used for tokenization purposes. The encoder model is constructed using a pre-trained BERT as a transformer. The decoder model uses LSTM, dense layers, and custom layers. To figure out the discrepancy between the expected probabilities and the actual binary labels, one uses the binary cross-entropy loss function. To reduce the loss, the Adam optimizer is employed. To enhance the deep learning-based models, experimental techniques, including word emphasis, data augmentation, and thresholding are used. Grid search hyperparameter tuning technique is used to find the optimal hyperparameters. The accuracy obtained for WiC-TSV dataset for BERT-LSTM model, BERT-LSTM model with word emphasis, BERT-LSTM model with data augmentation and word emphasis are 78.7%, 79.9%, and 80.6%, respectively. This implies BERT-LSTM model with data augmentation and word emphasis gives better accuracy for the WiC-TSV dataset.