Job Scheduling for Big Data Processing in YARN to Enhancing Performance

Digital data is growing as the network is growing. An enormous amount of data is gathered from various sources like social media sites, webpages, businesses, mobile data, etc. To analyze such massive data, one needs to have high computational resources and also the proper use of such resources. A single machine cannot handle the workload generated by such data. There may be a requirement for a group of machines that can work together to analyze such data. Many platforms provide ways to analyze such big data. One such is Hadoop YARN which has provisions for executing many concurrent applications at the same time. Hadoop YARN has a cluster to run applications and maximize resource utilization. It also provides a parallel programming paradigm. It has a distributed applications framework that provides job scheduling and monitoring. It also has big data storage with its processing capabilities and analysis of commodity hardware. The work performance of Hadoop can be improved as per requirements. For this, it provides over 200 parameter configuration settings. The improper configuration of these parameters may lead to the weak performance of the overall system, as finding a proper configuration becomes very important. In this paper, the various tuning parameters of Hadoop YARN are studied. A model is proposed for job scheduling in YARN that can maximize the performance of execution by implementing efficient job scheduling configurations. The proposed model is executed on a virtual machine Cloudera. The results show a decrease in processing time compared to the normal execution of the same program between the quantity demand and the price.