Blockchain technology is widely applied, yet existing data-acquisition methods, such as website API, full nodes, RPC, face limitations and struggle with exponential data growth. To address the challenges of obtaining blockchain transaction data, this paper proposes an optimized RPC-based ETL tool. The proposed method uses request-level parallel invocation, introduces synchronization semaphores, and incorporates path-compression techniques. A request-tracking tree is constructed, which generates SHA256-hashed fingerprints to distinguish node types. A union-find structure compresses its paths, reducing height and making the time complexity of the new request-record update O(1), improving the efficiency of the search and reducing the use of memory. Synchronization semaphores indicate incomplete requests, controlling execution order, avoiding resource contention, and ensuring accurate data processing. Experiments with Ethereum data show that it prevents data omission in collection, boosts efficiency, and enhances the reliability of blockchain data acquisition.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Optimizing Blockchain ETL Algorithms: Improvement Strategies Based on RPC Interfaces

  • Xiuwei Fan,
  • Dicen Wang,
  • Heng Qiu,
  • Licheng Weng,
  • Chunlong Sun

摘要

Blockchain technology is widely applied, yet existing data-acquisition methods, such as website API, full nodes, RPC, face limitations and struggle with exponential data growth. To address the challenges of obtaining blockchain transaction data, this paper proposes an optimized RPC-based ETL tool. The proposed method uses request-level parallel invocation, introduces synchronization semaphores, and incorporates path-compression techniques. A request-tracking tree is constructed, which generates SHA256-hashed fingerprints to distinguish node types. A union-find structure compresses its paths, reducing height and making the time complexity of the new request-record update O(1), improving the efficiency of the search and reducing the use of memory. Synchronization semaphores indicate incomplete requests, controlling execution order, avoiding resource contention, and ensuring accurate data processing. Experiments with Ethereum data show that it prevents data omission in collection, boosts efficiency, and enhances the reliability of blockchain data acquisition.