A Transformer-Based Stock Market Price Prediction by Incorporating BERT Embedding

The stock market trend is known to be volatile, dynamic, and nonlinear. Therefore, accurate prediction of the trend and forecasting the stock prices in today’s world is one of the most complex tasks. It is because of the events and preconditions, macro or micro, a few being politics, global economic conditions, and unexpected events which affect the stock market trend. Since it is difficult to predict all the contingencies, how long the effect of such parameters lasts can not be predicted. In this work, we studied the efficacy of different deep learning algorithms to learn the trend in the stock market price to predict the price for the next few days. We considered the stock price, stock index, dollar index, and related news data to predict the stock closing price of Apple Inc. Sentence embedding and sentiment scores were extracted from the news data and fed to the deep learning model along with stock price, stock index, and dollar index values. The deep learning model was designed using a Transformer consisting of an Encoder stack with attention layers and a set of MLP layers to reshape the predictions. The experiments showed that incorporating sentence embedding improved the prediction rate compared to the state-of-the-art model.