An Improvised Mechanism for Optimizing Fault Detection for Big Data Analytics Environment

In the applications of fault detection, the inputs are the data reflected from health state of the observed system. A major challenge to finding errors is the nonlinear relationship between the data. Big data have other drawbacks, and the volume and speed with which it is generated are reflected in the data streams themselves. In this paper, we develop a deep learning model that aims to provide fault detection in big data analytics engine. This investigation develops an approach for fault detection in large datasets using unsupervised learning. In this research, an unsupervised method of learning is developed specifically for the task of classifying large datasets. To discover regular textual patterns in large datasets, this research use data visualization methods. In this virtual environment, we employ an unsupervised learning method of machine learning that does not require human oversight. Instead, the system should be allowed some leeway to work and find things on its own. The unsupervised learning approach utilizes data that have not been tagged. In contrast to supervised learning, this approach can handle complex tasks.