Stock market prediction is a crucial task in finance, impacting investment decisions and market stability. This study proposes a novel approach combining fine-tuned Large Language Models (LLMs) with Transformer models to enhance stock price forecasting. LLMs analyze textual data for market trends, while the Transformer model predicts prices based on historical data using self-attention mechanisms to capture complex patterns and long-term dependencies. Experimental results show that this approach outperforms traditional models like LSTM and GRU, providing more accurate and interpretable predictions. This integrated method offers comprehensive insights into market dynamics, supporting robust decision-making for investors in complex market conditions.