CAAD: A Cognitive-Aware Framework for AI Agent Design in Complex Tasks
摘要
Large language model (LLM)-powered agents have demonstrated promising capabilities in assisting data analysis tasks. However, existing designs predominantly emphasize autonomous completion of predefined tasks rather than offering tailored support aligned with user needs. In complex, multi-phase analytical workflows, users frequently encounter diverse cognitive challenges-including vague task definitions, information overload, and decision-making uncertainty-that generic agent designs inadequately address. This paper presents the CAAD (Cognitive-Aware Agent Design) framework, a structured, stage-based design methodology grounded in cognitive load theory. CAAD decomposes the analytical process into four distinct phases: task definition, analysis, decision-making, and output. It guides the development of targeted agent interventions explicitly designed to reduce cognitive effort and enhance user experience. We applied the CAAD framework to a real-world case by analyzing task logs, conducting user interviews, and observing actual analytical behavior. Based on identified cognitive challenges, we developed specific support modules-including parameter selection cards, visual analytical aids, Chain-of-Thought (CoT) prompting for prioritization, and structured recommendation tools. Internal evaluation indicated that CAAD-based agent interventions reduced task completion time by 44.13% and significantly increased user confidence, as assessed through structured feedback sessions and surveys. Our findings underscore the necessity of aligning intelligent agent designs closely with users’ cognitive workflows. CAAD provides a generalized, transferable approach for designing explainable, adaptive agents particularly beneficial in high-stakes, cognitively demanding fields such as healthcare, finance, and scientific research.