The process of predicting stock price is required for wide ranging activities which include business decision-making, investment portfolio management, and wealth planning. But it is difficult to get accurate forecasts with the help of predictive modelling, and thus the selection of appropriate technique for different situations becomes quite important, taking into consideration the amount of past data available. This paper compares how effective four different approaches: the classic Autoregressive Integrated Moving Average (ARIMA), the supervised Random Forest (RF), the RNN-based Long Short-Term Memory (LSTM), and a hybrid model of LSTM with FinBERT sentiment analysis (LSTM + FinBERT) are for accurate predictions when the historical data is limited. All the models are trained and tested to check their ability to find market patterns, manage volatility, and give accurate forecasts with a small dataset. This paper tries to analyze advantages and limitations of these models, and how classical, machine learning, deep learning and sentiment analysis approaches are affected by shortages of data. Both regression metrics like Mean Squared Error (MSE), Root Mean Squared Error (RMSE), Mean Absolute Error (MAE), R-squared (R2) and directional metrics like accuracy were used for evaluation. The results show that Random Forest model performed the best, with high accuracy and R2 value greater than 0.94. However, other models failed when tested with unseen data, giving negative R2 values and large values of MSE, RMSE and MAE. This was seen due to severe overfitting for ARIMA, LSTM, and hybrid LSTM+FinBERT. Thus, it can be concluded by this research that when limited amount of historical data is available, ensemble methods like Random Forest are more reliable and accurate in their forecasts as compared to other more complex models which require a large amount of data.