Sentiment analysis in financial text plays a crucial role in understanding market trends and predicting financial outcomes. This research investigates the impact of fine-tuning and ensemble learning techniques on the performance of Large Language Models (LLMs) for financial sentiment analysis. We focus on comparing individual fine-tuned FinBERT models and various ensemble methods, particularly stacking ensembles with different meta-learners, on the FiQA (Financial Opinion Mining and Question Answering) 2018 benchmark dataset. Our methodology involves dataset selection, model development, and rigorous evaluation using multiple metrics. The results demonstrate the effectiveness of domain-specific adaptation and the potential benefits of combining multiple models in an ensemble to improve sentiment classification performance. The stacking ensemble with a Random Forest meta-classifier achieves state-of-the-art performance on both datasets, outperforming individual fine-tuned models and other ensemble methods.