A paradigm shift in higher education has been brought about by the rapid progress of large language models (LLMs), such as Open AI's GPT series. This shift could have an impact on teaching, learning, and evaluation approaches. In order to fully realize the potential of these technologies, this study aims to comprehensively analyze the body of literature already written about the use of LLMs in higher education settings, highlighting important areas of effect, pedagogical improvement opportunities, and obstacles that must be overcome. For this, scholarly publications, conference proceedings, and peer-reviewed papers were reviewed that addressed LLM integration in a range of educational contexts. A wide range of academic fields and instructional strategies were intended to be covered by the selection criteria, with an emphasis on research that discuss the implications of LLM use for teaching, learning, and assessment. According to research, LLMs can significantly improve education by facilitating individualized learning pathways, assisting with language learning, producing interesting materials, and encouraging higher-order thinking abilities. They do, however, also present difficulties, such as the spread of bias, threats to academic integrity, and the requirement that teachers adopt innovative teaching techniques. Due to their dual character, LLMs require the development of strategic ways for their integration that include ethical norms, pedagogical modifications, and ongoing effect assessment in order to maximize benefits and minimize risks.