In recent years, Sanskrit, like other languages, demands grammatical precision and adherence to traditional conventions. Often learned as a second language, Sanskrit writings can be susceptible to errors stemming from native language biases and unfamiliarity with its intricate rules. This paper proposes an automated system designed to identify and rectify such errors, offering support for both experienced and novice Sanskrit writers. Central to our investigation is the research question: Can Large Language Models (LLMs), effectively aid in the correction of contemporary Sanskrit writings, specially their contextual inference, where we struggle in rule-based and other statiscal approach? While the innate contextual inference capabilities of LLMs, driven by their transformer architectures, offer promise, the potential scarcity of Sanskrit-specific pre-training data poses a challenge. To address this, our approach evaluates the efficacy of various LLMs in grammar correction through prompt engineering, zero-shot inference, and few-shot inference, also delving into complementing rule-based engine. Although intuitively, we also delve into the relevance of both Encoder-Decoder Models (Seq2Seq Models) and Decoder Only Models (Autoregressive Models) for the task, exploring their strengths and weaknesses in the context of Sanskrit writing correction. However the scope of this work is limited to the later due to their huge size and availability.