A Genetic Algorithm Model for Join Order Query Optimization in Hadoop-Hive Framework

The continuous increase in data volume, measured in terabytes and zettabytes, has made big data an interesting area for researchers. Processing of queries in minimum time and cost is the main focus of researchers in the Hadoop-Hive Big Data Framework. Join is the most expensive and time-consuming operation in the query process because the join sequence’s complexity rises in proportion to the number of relations. There are N! ways to execute a specific query, where N is the number of relations in the join query. This indicates that the Hadoop-Hive framework’s query processing outcome is determined by the query optimizer’s search strategy. The foremost responsibility of a query optimizer is to find the optimal query execution plan (QEP) among various query plans in minimum time and cost. Various search strategies like deterministic strategies, randomized strategies, and evolutionary strategies have provided immense help in the direction of query optimization that has their pros and cons. Because both deterministic strategies and randomized strategies exhibit time and space complexity for generating optimal join order in minimum time for large join queries. This paper shall explore and comprehend evolutionary strategies for query optimization in Hive Hadoop Framework and shall propose a query optimization model using genetic algorithm as a powerful search strategy for generating optimal join order for solving large join queries.