Influence of Text Representations on the Efficacy of Classification Models Across Diverse Datasets
摘要
The efficacy of text classification models is significantly shaped by the choice of text representation, which critically enhances their interpretative capabilities. This study assesses the impact of various text representation methods, including Bag of Words (Unigram, Bigram, Trigram), TF-IDF, Word2Vec, GloVe, BERT (Base, Medium, and Large), DistilBERT, and notably, two innovative autoencoder-based approaches. The first autoencoder method generates embeddings from a combined dataset to create a universal representation that captures linguistic features common across multiple contexts. The second method involves training separate autoencoders for each dataset to develop tailored representations that address unique linguistic elements. These approaches were benchmarked across five datasets: Hate Speech and Offensive Language, SST-2, Fake News Classification, Cyberbullying, and IMDB, using classifiers such as linear regression, SVM, neural networks with three hidden layers, and LSTM. Performance was evaluated by accuracy and F1 score, highlighting substantial variations. BERT (Large) achieved remarkable F1-scores up to 0.998 and accuracies close to 99% in neural networks, demonstrating the robustness of dynamic, contextual embeddings. Autoencoder embeddings, customized for specific datasets, recorded F1-scores exceeding 0.970, showcasing their effectiveness in specialized contexts. These results underline the critical importance of selecting suitable text representations that synergize with dataset characteristics, offering insights into optimal model-dataset pairings and affirming the superiority of dynamic contextual methods over static ones in complex linguistic analyses.