Despite the availability of abundant digital data, there is a research gap in effectively utilizing this empirical evidence to inform the process of constructing or learning conceptual models. This gap highlights the need to address empirically-informed conceptual modeling practice. This paper underscores the potential of Artificial Intelligence (AI) for improving evidence-based conceptual modeling practice, specifically with Unified Modeling Language (UML) diagrams. We use generative AI to provide a look (visual) and sound (audio) perspective of UML diagrams. Our solution, named modSense, can link diagram elements to existing image and audio data or generate new image and audio content to improve understanding through real-world examples. Furthermore, our solution incorporates human preferences and feedback to dynamically adjust the generated or retrieved content to the user’s comprehension level, providing a tailored human-model interaction experience. Through increased engagement and real-world connections in UML diagrams, we aim to make models more aligned with business logic, resulting in better conceptual models and, subsequently, more effective computer programs. We report on the initial results of modSense and the modSense4All empirical study that focuses on assessing the educational impact of these multimodal resources in the domain of programming assistant software for surgery applications.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

How Does UML Look and Sound? Using AI to Interpret UML Diagrams Through Multimodal Evidence

  • Aleksandar Gavric,
  • Dominik Bork,
  • Henderik A. Proper

摘要

Despite the availability of abundant digital data, there is a research gap in effectively utilizing this empirical evidence to inform the process of constructing or learning conceptual models. This gap highlights the need to address empirically-informed conceptual modeling practice. This paper underscores the potential of Artificial Intelligence (AI) for improving evidence-based conceptual modeling practice, specifically with Unified Modeling Language (UML) diagrams. We use generative AI to provide a look (visual) and sound (audio) perspective of UML diagrams. Our solution, named modSense, can link diagram elements to existing image and audio data or generate new image and audio content to improve understanding through real-world examples. Furthermore, our solution incorporates human preferences and feedback to dynamically adjust the generated or retrieved content to the user’s comprehension level, providing a tailored human-model interaction experience. Through increased engagement and real-world connections in UML diagrams, we aim to make models more aligned with business logic, resulting in better conceptual models and, subsequently, more effective computer programs. We report on the initial results of modSense and the modSense4All empirical study that focuses on assessing the educational impact of these multimodal resources in the domain of programming assistant software for surgery applications.