Context. Many applications following a service-based architecture are regulated by Customer Agreements (CA) that define the conditions of use, responsibilities and limitations between suppliers and customers. There are tools for the automatic analysis of CAs that use Large Language Models (LLM) for some analysis operations. However, these tools do not take advantage of the Large Language Model Operations (LLMOps) cycle and best practices. Objective. Focusing on the need to automate the process of developing and operationalizing automatic analysis of CAs tools, this paper aims to explore the possibilities and limitations offered by LLMOps for this purpose. Method. To meet the proposed objective, we define an ideal LLMOps application for CAs analysers. In addition, a reference implementation has been designed that includes each phase of the LLMOps cycle. Results. After reviewing the automatic analysis of CAs tools and investigating the phases of the LLMOps cycle, an analysis and discussion of the alternatives for applying LLMOps for these tools has been conducted. Conclusions. From the eight stages of the LLMOps cycle, it can be concluded that some stages are more relevant than others. Ideally, the entire LLMOps process should be automated so that a new LLM can be incorporated in the future or an advanced feature of an LLM already in use can be activated.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Applying LLMOps to Support the Automatic Analysis of CAs

  • Pablo Landeta-López,
  • José María García,
  • Cathy Guevara-Vega,
  • Antonio Ruiz-Cortés

摘要

Context. Many applications following a service-based architecture are regulated by Customer Agreements (CA) that define the conditions of use, responsibilities and limitations between suppliers and customers. There are tools for the automatic analysis of CAs that use Large Language Models (LLM) for some analysis operations. However, these tools do not take advantage of the Large Language Model Operations (LLMOps) cycle and best practices. Objective. Focusing on the need to automate the process of developing and operationalizing automatic analysis of CAs tools, this paper aims to explore the possibilities and limitations offered by LLMOps for this purpose. Method. To meet the proposed objective, we define an ideal LLMOps application for CAs analysers. In addition, a reference implementation has been designed that includes each phase of the LLMOps cycle. Results. After reviewing the automatic analysis of CAs tools and investigating the phases of the LLMOps cycle, an analysis and discussion of the alternatives for applying LLMOps for these tools has been conducted. Conclusions. From the eight stages of the LLMOps cycle, it can be concluded that some stages are more relevant than others. Ideally, the entire LLMOps process should be automated so that a new LLM can be incorporated in the future or an advanced feature of an LLM already in use can be activated.