Purpose While Moodle is widely adopted in higher education, institutions struggle to leverage its features for adaptive learning. This study develops and validates the Adaptive Learning Readiness Assessment Framework (ALRAF), a course-level diagnostic instrument for evaluating a Moodle course’s structural capability to support adaptive learning experiences. Design We employ a quantitative cross-sectional design coupled with a novel Multi-LLM Synthetic Expert Consensus (MLSEC) protocol for content-validity evidence. ALRAF was developed through literature synthesis grounded in the ICAP framework (Chi and Wylie, Chi, Educational Psychologist49, 2014) and validated by a 40-panelist synthetic expert panel constructed across eight large-language-model providers and five stratified expert personas using two pre-registered Delphi rounds with falsifiable decision rules (<InlineEquation ID="IEq1"> <EquationSource Format="TEX">\(I\)</EquationSource> </InlineEquation>-CVI <InlineEquation ID="IEq2"> <EquationSource Format="TEX">\(\geq\)</EquationSource> </InlineEquation> 0.78, <InlineEquation ID="IEq3"> <EquationSource Format="TEX">\(V\)</EquationSource> </InlineEquation>-Aiken <InlineEquation ID="IEq4"> <EquationSource Format="TEX">\(\geq\)</EquationSource> </InlineEquation> 0.70, modified <InlineEquation ID="IEq5"> <EquationSource Format="TEX">\(\kappa^{*} \geq 0.74\)</EquationSource> </InlineEquation>). The validated framework was applied to a Moodle 3.8.2 dataset of 985 courses delivered at Kryvyi Rih State Pedagogical University (Ukraine) across 2020–2022. Findings The synthetic panel converged on six dimensions: Content Variety, Interaction Diversity, Assessment Flexibility, Learning Path Personalization, Feedback Mechanisms, and the panel-proposed AI & Data-Driven Adaptivity Integration (ADAI). The six-dimensional <InlineEquation ID="IEq6"> <EquationSource Format="TEX">\(\textrm{ALRS}_{6}\)</EquationSource> </InlineEquation> correlated significantly – but negatively – with the proportion of high grades (<InlineEquation ID="IEq7"> <EquationSource Format="TEX">\(A+B\)</EquationSource> </InlineEquation>): Pearson <InlineEquation ID="IEq8"> <EquationSource Format="TEX">\(r=-0.22\)</EquationSource> </InlineEquation>, and positively with low grades <InlineEquation ID="IEq9"> <EquationSource Format="TEX">\((E+F): r=+0.22\)</EquationSource> </InlineEquation> (both <InlineEquation ID="IEq10"> <EquationSource Format="TEX">\(p < .001\)</EquationSource> </InlineEquation>). This inverse relationship runs counter to what would be expected if ALRAF directly indexed pedagogical quality, and reframes ALRAF as a measure of structural readiness rather than learning effectiveness; we interpret the sign in Sect. <InternalRef RefID="Sec30">6</InternalRef> in terms of course-difficulty and compensatory-engineering effects. Faculty differences were significant (ANOVA <InlineEquation ID="IEq11"> <EquationSource Format="TEX">\(F(8, 976)=10.26\)</EquationSource> </InlineEquation>, <InlineEquation ID="IEq12"> <EquationSource Format="TEX">\(p < .001\)</EquationSource> </InlineEquation>, <InlineEquation ID="IEq13"> <EquationSource Format="TEX">\(\eta^{2}=0.078\)</EquationSource> </InlineEquation>, <InlineEquation ID="IEq14"> <EquationSource Format="TEX">\(\omega^{2}=0.070\)</EquationSource> </InlineEquation>). A multiple-regression model controlling for educational level, form of education, and faculty achieved adjusted <InlineEquation ID="IEq15"> <EquationSource Format="TEX">\(R^{2}=0.18\)</EquationSource> </InlineEquation> (<InlineEquation ID="IEq16"> <EquationSource Format="TEX">\(F(19, 965)=12.10\)</EquationSource> </InlineEquation>, <InlineEquation ID="IEq17"> <EquationSource Format="TEX">\(p < .001\)</EquationSource> </InlineEquation>). The framework reveals strong implementation of content variety but near-zero readiness in learning-path personalization and AI integration – itself a notable institutional finding. Contribution Methodologically, the study introduces MLSEC as a transparent AI-augmented approach to rubric content validation, with synthetic-panel limitations explicitly disclosed. Substantively, ALRAF provides a replicable structural-readiness index whose correlations with student outcomes are non-trivial in direction and magnitude; it helps institutions identify capability gaps (especially around AI integration) without claiming to forecast student success.

Development and validation of an adaptive learning readiness assessment framework for Moodle courses

期刊论文
出版模式：

开放获取
发表日期：

2026年6月16日

查看全文

Serhiy Semerikov,
Pavlo Nechypurenko,
Tetiana Vakaliuk,
Iryna Mintii,
Liliia Fadieieva

摘要

Purpose

While Moodle is widely adopted in higher education, institutions struggle to leverage its features for adaptive learning. This study develops and validates the Adaptive Learning Readiness Assessment Framework (ALRAF), a course-level diagnostic instrument for evaluating a Moodle course’s structural capability to support adaptive learning experiences.

Design

We employ a quantitative cross-sectional design coupled with a novel Multi-LLM Synthetic Expert Consensus (MLSEC) protocol for content-validity evidence. ALRAF was developed through literature synthesis grounded in the ICAP framework (Chi and Wylie, Chi, Educational Psychologist49, 2014) and validated by a 40-panelist synthetic expert panel constructed across eight large-language-model providers and five stratified expert personas using two pre-registered Delphi rounds with falsifiable decision rules ( \(I\) -CVI \(\geq\) 0.78, \(V\) -Aiken \(\geq\) 0.70, modified \(\kappa^{*} \geq 0.74\) ). The validated framework was applied to a Moodle 3.8.2 dataset of 985 courses delivered at Kryvyi Rih State Pedagogical University (Ukraine) across 2020–2022.

Findings

The synthetic panel converged on six dimensions: Content Variety, Interaction Diversity, Assessment Flexibility, Learning Path Personalization, Feedback Mechanisms, and the panel-proposed AI & Data-Driven Adaptivity Integration (ADAI). The six-dimensional \(\textrm{ALRS}_{6}\) correlated significantly – but negatively – with the proportion of high grades ( \(A+B\) ): Pearson \(r=-0.22\) , and positively with low grades \((E+F): r=+0.22\) (both \(p < .001\) ). This inverse relationship runs counter to what would be expected if ALRAF directly indexed pedagogical quality, and reframes ALRAF as a measure of structural readiness rather than learning effectiveness; we interpret the sign in Sect. 6 in terms of course-difficulty and compensatory-engineering effects. Faculty differences were significant (ANOVA \(F(8, 976)=10.26\) , \(p < .001\) , \(\eta^{2}=0.078\) , \(\omega^{2}=0.070\) ). A multiple-regression model controlling for educational level, form of education, and faculty achieved adjusted \(R^{2}=0.18\) ( \(F(19, 965)=12.10\) , \(p < .001\) ). The framework reveals strong implementation of content variety but near-zero readiness in learning-path personalization and AI integration – itself a notable institutional finding.

Contribution

Methodologically, the study introduces MLSEC as a transparent AI-augmented approach to rubric content validation, with synthetic-panel limitations explicitly disclosed. Substantively, ALRAF provides a replicable structural-readiness index whose correlations with student outcomes are non-trivial in direction and magnitude; it helps institutions identify capability gaps (especially around AI integration) without claiming to forecast student success.