An Intelligent Big Data Analysis Approach for Real-Time Data Processing: A Case Study on META Stock Price Prediction Using LSTM Model
摘要
Technological progress has made Big Data a major trend, posing significant challenges in managing vast datasets through digital technologies. These challenges include integrating diverse data, ensuring timely processing, and providing effective analysis and visualization. Big Data analytics uses AI, deep learning, and machine learning to uncover patterns and trends, improving decision-making and predictions. Real-time data processing faces additional difficulties related to handling high data velocity and volume, requiring robust and scalable infrastructure. This paper proposes an intelligent Big Data architecture for real-time data processing using Apache Kafka, the Elastic Stack, and Apache Spark. The approach is demonstrated through a case study predicting META stock prices using a Long Short-Term Memory (LSTM) model. By integrating deep learning, the architecture effectively addresses Big Data challenges, improving decision-making efficiency. The analysis uses stock data from Yahoo Finance, and the model’s performance was assessed using R-squared (R2) and Mean Absolute Error (MAE). The LSTM model achieved an R2 score of 0.99 and an MAE of 0.04, showing significant improvement over traditional models, particularly in handling volatile stock data.