Estimating Set Similarity Matrices for Link Prediction: An Empirical Analysis
摘要
There are several methods for finding set similarity and many different set similarity metrics (SSMs) have been proposed in the literature that serve specific applications. For example, the traditional SSM, such as the Jaccard index, is utilized for document deduplication in Web searches, whereas the Adamic–Adar index is used to predict links in social network analysis. However, while processing a large amount of data, it is very difficult to calculate the similarity between all possible pairs of sets involved in a given application scenario. This work aims to analyze efficient estimators for set similarity metrics. Firstly, the most relevant SSMs are introduced to understand the actual semantics of designing efficient estimators for these SSMs. Secondly, various estimators for SSMs (such as MinHash, SimHash, DotHash) are analyzed and summarized for state of the art. Finally, we have performed several experiments for link prediction tasks with these estimators by using real-world datasets: Facebook, Wikipedia, and Ogbl-ddi. Empirical results demonstrate that the DotHash estimator performs better than other estimators for link prediction tasks based on the Hits@K metric.