Large Language Models (LLMs) have significantly advanced Natural Language Processing (NLP), yet in low-resource languages like Marathi, such language models are underrepresented. This work deals with fine-tuning a large language model towards a Marathi language model to enhance better text summarization, question answering, and generation of texts. The pre-trained Transformer-based Language Model (MBART) was fine-tuned on the Marathi news dataset, hyperparameter-optimized, and then evaluated using ROUGE-L scores. For question-answering, the XLM-Roberta model was fine-tuned on a translated subset of SQuAD called MahaSQuAD and explored the influence of dataset size and training time. The Generative Pre-trained Transformer based text generation model (GPT)-2 was trained on a fraction of the L3Cube-MahaCorpus dataset but was trained further to generate more fluent content. This study explores how dataset quality, training duration, and hyperparameter choices influence model performance under varying computational constraints.