Enhancing Information Extraction from Building Standards with ChatGPT-4: A Multimodal Approach
摘要
The AECO (Architectural, Engineering, Construction, and Operations) industry is increasingly benefiting from the advantages of advanced information management, particularly through the use of Building Information Modeling (BIM) methods. BIM integrates geometric and alphanumeric data, such as dimensions and spatial relationships, into digital building models. These elements are subject to specific constraints defined by industry standards and guidelines. The issue with these constraints embedded in regulatory documents is that they are mostly only available in non-machine readable formats which hinders the exchange and usage of data within the building’s lifecycle. Currently, the required rules and requirements are extracted manually by experts from the regulatory documents and integrated manually and labor-intensively into corresponding BIM processes. Thus, many research projects are currently addressing the issue of how the textual components of regulatory documents can be automatically analyzed to extract rules and requirements using Natural Language Processing or other approaches. In addition to this textual information, many standards also contain figures and illustrations that explain the information described in the text in more detail or contain additional knowledge. Converting this graphical content into machine-readable information is just as challenging as analyzing the textual components. This paper aims to develop an approach for extracting information related to building information requirements from figures contained in standards using the two state-of-the-art MLLMs Kosmos 2 and Chat GPT-4. An MLLM is an AI system that can process text, images, audio, and video, unlike traditional models that are limited to text. MLLMs trained on large data sets recognize complex patterns in different types of data. With the help of the two MLLMs, the images, and their corresponding texts and captions from the standards will first be analyzed, and then the information obtained will be transferred into a structured data format to make it efficiently usable within BIM processes.