This paper evaluates the performance of three large language models (LLMs)—GPT-4, Claude, and Gemini—in retrieving and answering questions derived from construction regulatory documents. The case study focuses on the Sixth Book of the Aguascalientes Municipal Code, which outlines the technical requirements for building permits. A question-answering system was implemented using LangChain and tested with two formats of the source document: the original PDF and a manually reformatted plain-text version that was updated by an expert civil engineer. Embedding-based search (OpenAI and Google) and traditional TF-IDF retrieval methods were compared to assess the influence of retrieval strategy and document structure on overall model performance. The findings indicate that the format of documents plays a significant role in semantic retrieval. All models achieved higher accuracy when working with the reformatted text, with Google embeddings proving more sensitive to structural clarity. Gemini, when paired with Google embeddings and the plain-text format, produced the highest average similarity scores and the fastest response times. GPT-4 offered stable and competitive results in both formats, while Claude’s performance significantly improved with enhanced context but remained slower overall. Despite limitations in interpreting tabular data directly from PDFs, the models demonstrated strong potential when coupled with preprocessing strategies that linearize structured information. These findings suggest that LLMs, when correctly configured, can serve as valuable tools for navigating complex regulatory content. This study examines the integration of generative AI into technical and legal workflows, emphasizing its capabilities and the importance of careful retrieval and data preparation.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Automated Information Extraction from Construction Regulations Using LangChain: A Case Study in Aguascalientes

  • Claudia Angelica Limon Flores,
  • Aurora Torres Soto,
  • María Dolores Torres Soto,
  • Fredicio Soledad Reyes

摘要

This paper evaluates the performance of three large language models (LLMs)—GPT-4, Claude, and Gemini—in retrieving and answering questions derived from construction regulatory documents. The case study focuses on the Sixth Book of the Aguascalientes Municipal Code, which outlines the technical requirements for building permits. A question-answering system was implemented using LangChain and tested with two formats of the source document: the original PDF and a manually reformatted plain-text version that was updated by an expert civil engineer. Embedding-based search (OpenAI and Google) and traditional TF-IDF retrieval methods were compared to assess the influence of retrieval strategy and document structure on overall model performance. The findings indicate that the format of documents plays a significant role in semantic retrieval. All models achieved higher accuracy when working with the reformatted text, with Google embeddings proving more sensitive to structural clarity. Gemini, when paired with Google embeddings and the plain-text format, produced the highest average similarity scores and the fastest response times. GPT-4 offered stable and competitive results in both formats, while Claude’s performance significantly improved with enhanced context but remained slower overall. Despite limitations in interpreting tabular data directly from PDFs, the models demonstrated strong potential when coupled with preprocessing strategies that linearize structured information. These findings suggest that LLMs, when correctly configured, can serve as valuable tools for navigating complex regulatory content. This study examines the integration of generative AI into technical and legal workflows, emphasizing its capabilities and the importance of careful retrieval and data preparation.