The operational treatment of sensitive textual data requires efficient natural language processing techniques running on local hardware. This paper investigates the possibility of using Large Language Models (LLMs) to extract semantic relations from specialized corpora, particularly drug trafficking language, not directly, but by generating Finite-State Transducers (FSTs) for the NooJ platform for local execution. In that hybrid approach, LLMs construct NooJ grammars and dictionaries, later compiled into deterministic NooJ automata, which run fast on normal PCs. Experiments on a corpus of 75 text files (164,000 tokens, 27,000 distinct words) highlight challenges in alias recognition and structured evaluation importance. Results show that while direct LLM extraction produces noisy outputs, to use an LLM to automatically produce a corresponding NooJ “grammar” (FST + dictionary), which can then be improved by human experts, is a distinct possibility for handling specialized sublanguages.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Use of LLMs for Semantic Relation Extraction and Development of NooJ Lingware Applied to Drug Trafficking Language

  • Nicolas Boffo,
  • Mathieu Lafourcade,
  • Christian Boitet

摘要

The operational treatment of sensitive textual data requires efficient natural language processing techniques running on local hardware. This paper investigates the possibility of using Large Language Models (LLMs) to extract semantic relations from specialized corpora, particularly drug trafficking language, not directly, but by generating Finite-State Transducers (FSTs) for the NooJ platform for local execution. In that hybrid approach, LLMs construct NooJ grammars and dictionaries, later compiled into deterministic NooJ automata, which run fast on normal PCs. Experiments on a corpus of 75 text files (164,000 tokens, 27,000 distinct words) highlight challenges in alias recognition and structured evaluation importance. Results show that while direct LLM extraction produces noisy outputs, to use an LLM to automatically produce a corresponding NooJ “grammar” (FST + dictionary), which can then be improved by human experts, is a distinct possibility for handling specialized sublanguages.