Generative AI-Based Automated Assessment Generation
摘要
This chapter presents the application of Generative AI (GenAI) for Automatic Assessment Generation (AAG) in STEM education, with a focus on contextual and culturally responsive approaches. It presents two in-depth case studies that illustrate the development and application of GenAI-enabled AAG tools, organized around five critical components: research focus, GenAI configuration, item generation strategies, human involvement, and quality control measures. The first case introduces the Culturally Responsive Science Assessment Generator (CRSciA), a customized GPT-based tool that integrates culturally responsive pedagogical frameworks and interactive, dynamic prompts. This tool is designed to generate K-12 science assessment items that reflect students’ cultural and linguistic backgrounds across multiple national contexts, while aligning with established educational standards. The second case offers an empirical investigation comparing the quality of scenario-based medical physiology questions generated by ChatGPT to those developed by experienced faculty. The analysis reveals key differences in psychometric properties, including difficulty, discrimination, and distractor effectiveness. These cases demonstrate the potential of prompt engineering and customized large language models to streamline the creation of assessments. However, the chapter also underscores the limitations of relying solely on AI-driven approaches and advocates for future research to prioritize human-in-the-loop methodologies. These approaches should strategically integrate GenAI’s efficiency with the pedagogical insight and cultural awareness of human educators to produce high-quality, equitable, and instructionally meaningful STEM assessments.