Transition-Based Heterogeneous Network Embedding by Self-attention Model of Sparse Matrix

Heterogeneous network embedding aims to embed the networks into low dimensional spaces, in which each vertex is represented as a low-dimensional vector. Compared with homogeneous graphs, heterogeneous graphs increase the complexity of representation learning due to their heterogeneity. In this paper, we distinguish different types of edges in heterogeneous networks and project the nodes into different relational spaces. In addition, all previous works on heterogeneous graph representation ignored the imbalance in heterogeneous networks. Inspired by the TranSparse method used in the field of knowledge graph embeddings, we extend the idea of this method to heterogeneous networks to handle the imbalance. We consider replacing the transition matrix used in projection with sparse-matrices-correlation matrix, where the sparse matrices used are adaptive. For the two sparse patterns (structured and unstructured), we propose to combine them using a self-attention model, which effectively integrates the information extracted from the two patterns. Experiments are conducted on three heterogeneous networks to evaluate our algorithm on link prediction tasks and the experimental results show that our algorithm outperforms other baseline algorithms for heterogeneous network embeddings.