Design and Validation of the Psychometric Properties of a Questionnaire Measuring AI Literacy Among University Students: Preliminary Results
摘要
The rapid adoption of digital technologies and artificial intelligence (AI) necessitates equipping university students with AI literacy. Since the rise of Generative AI in 2020, frameworks have emerged to define key competencies, spanning technical knowledge, ethical considerations, and AI applications. This paper presents an overview of an ongoing research project assessing undergraduate students’ perceived self-efficacy in AI literacy in Quebec. Specifically, it focuses on the design and validation of the questionnaire’s psychometric properties. The study follows a rigorous methodological approach, grounded in a scoping review of AI literacy frameworks, which identified five key dimensions: Technical aspects, Ethical aspects, Critical thinking, AI use and Human agency. Study 1 involved 768 students completing an initial questionnaire in 2023. It encompassed three key dimensions—Technical aspects, Ethical considerations, AI use and general computer literacy. Exploratory factor analysis (EFA) with Varimax rotation validated a four-factor structure, explaining 57.5% of variance across 28 items. Study 2 expanded the questionnaire to integrate dimensions not covered in Study 1 (AI use for learning, Critical thinking, Human agency). Twelve items from three scales in Study 1 were retained, while 34 additional items were added to ensure comprehensive coverage of all dimensions identified in the scoping review. The revised questionnaire is being administered at two universities in Quebec and will be subject to exploratory and confirmatory factor analyses. The findings will be presented at the upcoming conference in July.