Sentiment analysis is a powerful tool that decodes emotions in textual data, providing invaluable insights for businesses to understand customer sentiments and market trends. Despite their utility, current sentiment analysis models often fall short in specialized domains, particularly in interpreting the complex language of the financial sector. Addressing this gap, our research presents Gemma 7B, a fine-tuned language model specifically enhanced for financial sentiment analysis. We enrich the model with subword regularization to better interpret financial sentiment and domain-specific tokenization to grasp the subtle differences of market-related discourse. Our approach is a hybrid, leveraging the best of both lexicon-based and machine learning methodologies, thus paving the way for a more nuanced and accurate analysis. The results are compelling: before fine-tuning, Gemma 7B achieved an overall accuracy of 73.2% in sentiment classification. Post-fine-tuning, the model’s accuracy improved to 97.8%, with a particularly notable improvement in neutral sentiment detection—from a meager 19.3% to an impressive 87.7%. These figures not only demonstrate Gemma 7B’s enhanced capability but also signify a substantial advancement over traditional sentiment analysis models in processing complex financial texts. This research contributes a novel fine-tuning strategy that significantly refines sentiment analysis in the financial realm. The robust performance of Gemma 7B post-fine-tuning underscores the potential of our specialized model in transforming the landscape of financial sentiment analysis. Our future endeavors will focus on further refining this model to capture real-time shifts in market sentiment, fostering more strategic and data-driven decision-making in finance.