This paper investigates the effectiveness of self-supervised pretraining in the context of Forex price prediction. Specifically, we explore whether pretraining a model using Masked Time Modeling (MTM) can enhance the forecasting performance of an Long Short-Term Memory (LSTM) network compared to a baseline trained solely on labeled data. The task involves predicting the next five closing prices of a currency pair based on the previous 50 observations. We conducted experiments on five currency pairs, each tested across three different time intervals. The MTM-based model was first pre-trained to reconstruct masked price sequences, learning contextual representations of the time series. These representations were then used as input features in a fine-tuned LSTM model trained for supervised forecasting. Both models shared the same architecture, and their performance was evaluated using the Root Mean Squared Error (RMSE). The results show that self-supervised pretraining with MTM led to slightly better performance for some currency pairs, but significantly worse results for others compared to the purely supervised model.