CRAT: A Cross Reference Attention Transformer Model to Extract Domain Relevant Concepts from the Student Lecture Notes

A major challenge in the educational process is the volume of information students must consolidate from extensive class notes. Key concept extraction addresses this by allowing students to focus on meaningful and relevant content. This paper introduces Cross Reference Attention Transformer (CRAT), a novel model that employs token level and sentence level multi-head attentions concurrently to extract key concepts based on domain knowledge. The goal is to provide concise, logically structured, and contextually relevant content, tailored to the student’s academic needs. The proposed model was tested on collected classroom lecture note datasets across various academic levels, including Grade X Social Science, Grade XII English, BBA, BSW and MSW different topics, and evaluated using standard metrics like ROUGE and METEOR. It also tested on existing datasets of Webis-TLDR_17, PubMed Abstracts and Scientific Papers. The results demonstrate that CRAT outperforms existing models, achieving a 20% average improvement in ROUGE-1 scores and a 12% average enhancement in METEOR compared to PEGASUS, BERTSUMABS, and other baselines. This work presents a domain-aware key concept extraction model that enhances content relevance, making it particularly suitable for processing challenging academic materials by incorporating domain-specific knowledge into the extraction process.