Enormous quantities of data are generated every second by the emergence of ubiquitous gadgets and the related new technologies. With such an enormous quantity of redundant data, processing and storing it becomes exceedingly difficult. The Content Defined Chunking (CDC) approach is crucial in reducing data duplication due to its high redundancy. These issues are addressed in this study by presenting a novel technique that can reduce redundant data and improve storage efficiency. In order to determine the appropriate breakpoints between chunks and choose the most frequent sequences of adjacent bytes as breakpoints, the study suggests a novel method that uses a set of Identical Patterns (IP). The advanced three-layered hashing (ATLH) technique has been used to reduce the processing expense as well as the additional storage needs for the hashing function. In a comprehensive experiment, the effectiveness of the suggested system was assessed using the various data sets. The suggested approach tries to improve hashing performance and decrease the amount of storage needed for the index table.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Novel Data Deduplication Technique Using Content-Defined Chunking

  • A. Sevuga Pandian,
  • M. Gomathi

摘要

Enormous quantities of data are generated every second by the emergence of ubiquitous gadgets and the related new technologies. With such an enormous quantity of redundant data, processing and storing it becomes exceedingly difficult. The Content Defined Chunking (CDC) approach is crucial in reducing data duplication due to its high redundancy. These issues are addressed in this study by presenting a novel technique that can reduce redundant data and improve storage efficiency. In order to determine the appropriate breakpoints between chunks and choose the most frequent sequences of adjacent bytes as breakpoints, the study suggests a novel method that uses a set of Identical Patterns (IP). The advanced three-layered hashing (ATLH) technique has been used to reduce the processing expense as well as the additional storage needs for the hashing function. In a comprehensive experiment, the effectiveness of the suggested system was assessed using the various data sets. The suggested approach tries to improve hashing performance and decrease the amount of storage needed for the index table.