The development of web applications often requires the translation of design mock-ups into functional code, a process that is time-consuming and inaccessible to those without technical expertise. This research addresses this challenge by leveraging Large Language Models (LLMs) to automate the derivation of IFML-like Graphical User Interface (GUI) models from mock-up images. These models serve as a foundation for the validation and subsequent generation of web applications using available deterministic code-generators. By enhancing the BESSER low-code framework with this extension, we provide an advanced pipeline that integrates LLMs into model-driven web application development relying on the UML (for the structural models) and IFML (for the GUI ones) standards. Our LLM-based approach achieves high accuracy for both digital and hand-drawn mock-ups, demonstrates efficiency with rapid run-time performance, and delivers web pages that have a high structural similarity to their corresponding mock-ups. The complete infrastructure is available in an open-source repository.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

From Mock-Ups to IFML-Like GUI Models: Using Large Language Models in Web Engineering

  • Atefeh Nirumand,
  • Jordi Cabot

摘要

The development of web applications often requires the translation of design mock-ups into functional code, a process that is time-consuming and inaccessible to those without technical expertise. This research addresses this challenge by leveraging Large Language Models (LLMs) to automate the derivation of IFML-like Graphical User Interface (GUI) models from mock-up images. These models serve as a foundation for the validation and subsequent generation of web applications using available deterministic code-generators. By enhancing the BESSER low-code framework with this extension, we provide an advanced pipeline that integrates LLMs into model-driven web application development relying on the UML (for the structural models) and IFML (for the GUI ones) standards. Our LLM-based approach achieves high accuracy for both digital and hand-drawn mock-ups, demonstrates efficiency with rapid run-time performance, and delivers web pages that have a high structural similarity to their corresponding mock-ups. The complete infrastructure is available in an open-source repository.