Building Blocks

Our community detection construction is intended to be applied to large social networks where the objects are individuals and the connections are based on the similarity of their personalities, tastes, opinions, and online activity. We separate the attribute-attribute similarity between individuals into three different modalities: their similarity in textual content, their similarity in hashtag use, and their similarity in the content of images posted. Sophisticated extraction is required for these last two to discover the words that make up hashtags, and what images are ‘about’. We also use the explicit links between individuals created by one ‘following’ another, a structural similarity. Spectral graph embedding is one way to reduce the complexity of finding communities. It maps a graph (representing similarities in each of the four modalities) into a geometric embedding. In the resulting embedding, any geometric clustering technique can be used to find communities. We introduce DBSCAN as one solution. A large number of measures have been defined for measuring the quality of communities. We define and compare these measures which we will use later. The semantic similarity of the members of any detected community can be assessed by using topic modelling to describe the content, in its broadest sense, created by its members. Three common methods for topic modelling are Latent Semantic Indexing, Latent Dirichlet Allocation, and Non-negative Matrix Factorisation.