Pretesting of Survey Instruments: Methodology and Results
摘要
This chapter outlines the methodology and results of the pretesting phase conducted to refine the survey instruments used for evaluating the integration of generative AI (GenAI) in the master’s courses M562 and M563. Pretesting was a crucial step to ensure that the immediate and follow-up evaluation questionnaires met high standards across several dimensions: clarity and comprehension of questions, appropriateness and acceptability of response options, impartiality and lack of bias, overall quality of the questionnaire, clarity and sufficiency of instructions, logical flow of questions, and time required to complete the survey. By involving a diverse group of master’s students and using both quantitative feedback and follow-up interviews, the process identified ambiguities, cultural nuances, and practical improvements to strengthen the instruments. The refinements and insights presented in this chapter aim to support peers who wish to reuse or adapt the pretesting approach and survey instruments for similar educational interventions involving GenAI or related technologies.