Separating DNA Sequences of Microbes Using Apache Spark

Metagenomic projects focus on analyzing microbial communities which contain a huge amount of DNA sequences. Therefore, utilizing high performance computing for analysis of microbial data is essential to enhance data processing efficiency. This study proposes a parallelized approach based on Apache Spark technology for clustering sequences of microbes in which DNA sequences are separated into groups of different organisms. The proposed method is derived from an available algorithm and designed to exploit the strength of high performance systems. Experimental results demonstrate the effectiveness of the proposed method which achieves significantly improved processing speed compared to its single mode algorithm.