This study examines the cognitive dynamics of collaborative design by comparing human-AI collaborative environments with traditional sketching through multimodal analysis. By integrating eye-tracking data and think-aloud protocol coding, we investigated transitions between cognitive states (Problem Solving, Evaluation, Operation, and Naivety) during the creative process. Four design experts with architectural backgrounds participated in a mixed within/between-subject experiment. Results reveal distinct patterns: human-AI collaborative design promotes prolonged fixation and evaluation-driven cognitive cycles, fostering reflective interpretation of AI outputs; traditional design encourages exploratory visual behavior and direct shifts from ideation to execution. Transition matrices quantify these differences, demonstrating how collaborative AI systems reshape designers’ cognitive rhythms. These findings provide empirical foundations for developing more cognitively aligned human-AI collaborative systems and offer a novel framework for analyzing human-AI interaction in creative contexts.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Multimodal Analysis of Human-AI Collaborative Design: Eye-Tracking and Think-Aloud Insights into Cognitive Transitions

  • Chor-Kheng Lim

摘要

This study examines the cognitive dynamics of collaborative design by comparing human-AI collaborative environments with traditional sketching through multimodal analysis. By integrating eye-tracking data and think-aloud protocol coding, we investigated transitions between cognitive states (Problem Solving, Evaluation, Operation, and Naivety) during the creative process. Four design experts with architectural backgrounds participated in a mixed within/between-subject experiment. Results reveal distinct patterns: human-AI collaborative design promotes prolonged fixation and evaluation-driven cognitive cycles, fostering reflective interpretation of AI outputs; traditional design encourages exploratory visual behavior and direct shifts from ideation to execution. Transition matrices quantify these differences, demonstrating how collaborative AI systems reshape designers’ cognitive rhythms. These findings provide empirical foundations for developing more cognitively aligned human-AI collaborative systems and offer a novel framework for analyzing human-AI interaction in creative contexts.