Software projects in the medical device domain specify requirements at different abstraction levels (layers) to ensure traceability, compliance, and clarity. However, writing detailed lower-level requirements is time-consuming. Privacy and regulatory constraints often prohibit the use of external or public cloud services for processing sensitive requirement data. This restriction motivates the need for evaluating on-premise LLMs, which, to our knowledge, have not yet been studied in this context. This study investigates whether privacy-preserving, on-premise large language models (LLMs) can automate the decomposition of high-level requirements into system and software-level specifications while complying with data-protection regulations. Five open-weights instruction-tuned models ranging from 3 billion to 70 billion parameters are evaluated locally using the Ollama runtime. Four prompt strategies are tested: minimal, regulatory-context, example-driven, and retrieval-augmented generation (RAG), across two decomposition levels: user-to-system and system-to-software on real-world medical device requirements. The results indicate that (i) all on-premise models generate syntactically valid JSON and correctly structured requirements when prompted appropriately, (ii) example-driven prompts achieve the highest semantic similarity scores to the ground truth, (iii) larger models (R1 Distill Qwen 32B and LLaMA 3.3 70B) outperform smaller models, and (iv) RAG, which is used to fetch examples for few-shot prompting, shows no measurable benefit due to limited retrieval corpus. These findings demonstrate that local LLMs can effectively automate requirements decomposition while maintaining regulatory compliance. The Proposed approach has the potential to reduce the time used in requirements engineering while maintaining regulatory compliance.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Enhancing Regulation-Adherent Requirement Engineering with Contextual AI: An Industrial Study

  • Orhan Sirin,
  • Malik Abdul Sami,
  • Tuomas Granlund,
  • Jussi Rasku,
  • Zheying Zhang,
  • Pekka Abrahamsson

摘要

Software projects in the medical device domain specify requirements at different abstraction levels (layers) to ensure traceability, compliance, and clarity. However, writing detailed lower-level requirements is time-consuming. Privacy and regulatory constraints often prohibit the use of external or public cloud services for processing sensitive requirement data. This restriction motivates the need for evaluating on-premise LLMs, which, to our knowledge, have not yet been studied in this context. This study investigates whether privacy-preserving, on-premise large language models (LLMs) can automate the decomposition of high-level requirements into system and software-level specifications while complying with data-protection regulations. Five open-weights instruction-tuned models ranging from 3 billion to 70 billion parameters are evaluated locally using the Ollama runtime. Four prompt strategies are tested: minimal, regulatory-context, example-driven, and retrieval-augmented generation (RAG), across two decomposition levels: user-to-system and system-to-software on real-world medical device requirements. The results indicate that (i) all on-premise models generate syntactically valid JSON and correctly structured requirements when prompted appropriately, (ii) example-driven prompts achieve the highest semantic similarity scores to the ground truth, (iii) larger models (R1 Distill Qwen 32B and LLaMA 3.3 70B) outperform smaller models, and (iv) RAG, which is used to fetch examples for few-shot prompting, shows no measurable benefit due to limited retrieval corpus. These findings demonstrate that local LLMs can effectively automate requirements decomposition while maintaining regulatory compliance. The Proposed approach has the potential to reduce the time used in requirements engineering while maintaining regulatory compliance.