Comparative analysis of sentence similarity detection using machine and deep learning with vectorization techniques
摘要
Sentence similarity detection provides significant advantages across different applications, such as customer support applications, e-commerce customer service, educational platforms, community opportunities and question-answering systems. This study presents a comparative analysis of various machine learning and deep learning models for sentence similarity detection, including cosine similarity, adaptive boosting (AdaBoost), Extreme Gradient Boosting (XGBoost), Convolutional Neural Network with Long Short-Term Memory, and Bidirectional Encoder Representations from Transformers with Long Short-Term Memory. This research also evaluates the impact of various vectorization techniques, such as Term Frequency-Inverse Document Frequency, OpenAI embeddings and Topic Modeling, on the performance of these models. The proposed research validates the effectiveness of these approaches in enhancing the accuracy of similarity detection. The findings offer valuable insights into an optimal combination of vectorization methods and models for improved sentence similarity detection.