Using Similarity Based on Embeddings

So far we have used a simple metric for the similarity between the lists of words used by pairs of users: their Jaccard similarity. Jaccard similarity counts the overlap between two lists of words but it is purely syntactic: two words are either identical and match, or do not match. Often words can look syntactically different but have similar meanings, so better calculations of user-user similarity can be made by considering the semantics of words. A popular way to do this is to use an embedding derived from neural network techniques, where each word is embedded in a vector space in which closeness represents semantic similarity. Word similarity can be naturally lifted to list-of-word similarity within the same framework, providing a more powerful way to measure how similar two lists of words are. We use BERT which has the added advantage that it has a builtin topic modelling capability. We show that there are some changes to the communities detected but they are relatively small. Finally, we apply a Large Language Model, GPT3, to the content of each cluster and extract a qualitatively different description of the content of the cluster.