This paper focuses on the innovative application of Large Language Models (LLM) and Retrieval-Augmented Generation (RAG) question-answering architectures in the context of data classification and grading (DCG) within universities. Given the explicit requirements of the “Data Security Law of the People's Republic of China” for a DCG system, this paper proposes a method that integrates the text understanding and generation capabilities of LLM with the efficient retrieval mechanism of RAG, aiming to enhance the accuracy and efficiency of DCG in universities. By delving into the advantages of LLM, the working principles of RAG, and their specific applications in DCG, this paper demonstrates how RAG can effectively overcome the limitations of traditional models, presenting a novel and precise solution for data governance in universities. It also emphasizes the importance of combining technological applications with legal and regulatory requirements.