Exploring the transformative potential of large language models (LLMs) in revolutionizing the assessment process in education is a pressing need. LLMs have the capability to automatically evaluate student submissions, significantly enhancing the educational landscape and providing a second opinion and additional support to human evaluators. Therefore, our evaluation of various LLMs aims to assess their ability to provide automated, detailed, consistent, unbiased, and efficient feedback and scoring in educational settings, instilling a sense of optimism about the future of assessment. To achieve our objective, we comprehensively evaluated various state-of-the-art LLMs, such as GPT-4, GPT-4o, and Mixtral 8x22B on diverse datasets, including ASAP SAS, ASAP AES, and a real-world BWD dataset specifically collected for this study. The experimental results on these datasets employing sound prompt engineering techniques demonstrate that LLMs possess the potential not only to automate the scoring of student submissions but also to accurately match the scores of human assessors for actual courses taught in universities. Notably, GPT-4o exhibited promising capabilities in scoring short-answer submissions. These models were particularly good in STEM-related domains for tasks with clear structure, well-defined rubrics, and minimal subjective interpretation. However, the study identifies specific challenges for certain tasks, particularly creative tasks, underscoring the need for further research in this area.