Stock market prediction is challenging due to the very complex and volatile nature of financial markets. Traditional approaches often struggle to capture the intricate models and dependencies within stock price data. Machine learning techniques, particularly Recurrent Neural Networks (RNNs), have shown promise in modeling sequential data, making them suitable for stock market prediction. This paper explores the application of Guided Stochastic Gradient Descent (GSGD), an optimization technique, combined with guided training, to improve the performance of Recurrent Neural Networks (RNNs) in predicting stock price movements. We refer to this enhanced model as Guided RNN (GRNN). Data from Yahoo Finance, covering 15 years and including only prices, was used for this study. The performance of GRNN was compared with standard RNN and Long Short-Term Memory (LSTM) networks. The evaluation was based on accuracy, F1 score, ROC-AUC, and loss curves. The results demonstrate that GSGD significantly improves the predictive performance of RNNs, handling inconsistent data effectively and revisiting batches based on loss. Comparisons with other studies published in the last five years highlight the advantages of GRNN. Future work includes exploring the integration of GRNN with LSTM models.