Trustworthy AI in Design: Introducing Explainable Agent Systems
摘要
As industrial product development becomes increasingly complex and knowledge-intensive, the integration of Artificial Intelligence (AI) agents into design workflows offers great potential to improve efficiency and decision making. However, the opacity of current AI reasoning processes remains a major obstacle for adoption in engineering domains. This position paper explores the need for Explainable AI (XAI) within agentic design systems, proposing a conceptual architecture where agents, powered by Large Language Models (LLMs), not only perform domain-specific tasks, but also generate human-readable justifications for their decisions. Unlike black-box systems, these agents are designed to promote transparency, trust, and traceability, all of which are critical in high-stakes industrial contexts. Building upon the foundation of the Agentic Approach to Product Design, we outline how roles such as requirement analysis, material selection, and specification interpretation can be reimagined with explainability at their core. This work advocates for a shift towards interpretable, auditable AI assistants, capable of supporting collaborative engineering processes. An illustrative scenario is used to exemplify the practical value and challenges of agents supported by XAI. Future research directions are highlighted, including evaluation metrics for explainability and potential integrations into existing agent orchestration platforms such as CrewAI. As a conceptual position paper, this work aims to stimulate the development of explainable multi-agent design systems and guide future empirical validation in industrial contexts.