An Investigation of Activation Functions and Hyperparameters in Long Short-Term Memory Network Architectures for Financial Time Series Forecasting
摘要
This paper examines the architecture options of long short-term memory networks in the problem of forecasting financial time series using the example of cryptocurrency prices on minute, hourly, and daily ticks. The aim of the work was to study the influence of the activation function and hyperparameters of the long short-term memory network on forecast quality. Classic activation functions such as sigmoid, hyperbolic tangent, linear, rectified linear unit, as well as the nonclassical activation functions cloglog and loglog are considered. Hyperparameters define the architectural decisions of the neural network and affect the quality of the forecast and the speed of learning. This paper shows that the correct choice of the activation function and hyperparameters of long short-term memory networks is an important factor that influences the forecast quality and that the nonclassical activation functions cloglog and loglog should be used in long short-term memory network architectures for time series forecasting tasks along with classical activation functions.