Unified stream and batch computing (USBC) aims to incorporate stream and batch computation into a unified framework, thereby enabling the development of a one-stop solution for stream and batch data processing and enhancing the generalization of the framework. However, research on unified graph computing models (UGCMs) faces several challenges. First, existing UGCMs need to consider all graph information in the cache during the incremental update phase, thus leading to decreased execution efficiency. Second, existing UGCMs use fixed bytes to store nodes without considering the actual space occupied by nodes resulting in wasted memory when dealing with large graphs. This paper proposes a UGCM with Local Updates for community detection (UGCM-LU). We first implement a local update strategy to consider partial information of the graph to achieve incremental updates. Secondly, we also designed a byte-compression-based module to store graph data according to the space occupied by nodes. The experimental results show the effectiveness and efficiency of the model in real-world and artificial networks.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

UGCM-LU: A Unified Stream and Batch Graph Computing Model with Local Update for Community Detection

  • Hong Li,
  • Ling Wu,
  • Kun Guo

摘要

Unified stream and batch computing (USBC) aims to incorporate stream and batch computation into a unified framework, thereby enabling the development of a one-stop solution for stream and batch data processing and enhancing the generalization of the framework. However, research on unified graph computing models (UGCMs) faces several challenges. First, existing UGCMs need to consider all graph information in the cache during the incremental update phase, thus leading to decreased execution efficiency. Second, existing UGCMs use fixed bytes to store nodes without considering the actual space occupied by nodes resulting in wasted memory when dealing with large graphs. This paper proposes a UGCM with Local Updates for community detection (UGCM-LU). We first implement a local update strategy to consider partial information of the graph to achieve incremental updates. Secondly, we also designed a byte-compression-based module to store graph data according to the space occupied by nodes. The experimental results show the effectiveness and efficiency of the model in real-world and artificial networks.