Evaluating Blended Learning: Instructional Design Case Studies

Evaluation happens at different stages of courses, projects and programs. At the outset, data informs the ideation and conceptualization phase. Next, during the building process, evaluation techniques steer the design and implementation. Once deployed, many evaluators provide formative data at the midway point, which allows for course correction. Finally, at the conclusion of a project or course, summative data is used to ascertain that lessons learned don’t become lessons lost. The case studies in this chapter illustrate various aspects of evaluation in blended settings. They offer (partial) solutions to technological, methodical and epistemological challenges. They are embedded in a literature review that delineates general approaches to evaluation research, relevant theoretical frameworks in education, as well as standards and guidelines for online and blended learning settings. The literature section explores diverse theoretical-methodological frameworks that underpin the evaluation of technology in education. It considers Activity Theory, Action Research, Scholarship of Teaching and Learning (SoTL), Design-Based Research (DBR), Pattern Languages, Technological Pedagogical Content Knowledge (TPACK), and the Community of Inquiry (CoI) as foundational perspectives. Each framework offers a unique lens through which the efficacy of educational technology can be analyzed, from understanding human interaction with tools in a socio-cultural context to iterative design and implementation of educational strategies. The literature review further provides a concise overview of key standards and guidelines pertinent to online and blended learning, focusing on Universal Design for Learning (UDL), Quality Matters (QM), usability, and Web Content Accessibility Guidelines (WCAG). Finally, the empirical case studies are a curated selection of evaluation examples spanning 20 years of evaluating educational technologies, courses, and programs, with a focus on the theme of ‘blending.’ The cases illustrate diverse challenges in the field of instructional design, ranging from the integration of standards and guidelines in online program development to the evaluation of open learning environments and the strategic consolidation of organizational data.