Testbed for Evaluating LLMs on Concept Naming Within Ontology Transformation
摘要
Given the demand for applying knowledge graphs and ontologies in diverse kinds of applications and contexts, it is highly desirable to assure their structural flexibility, such that the same semantic content could be expressed via slightly different patterns and idioms. The key to such flexibility is the capability to execute structural transformation at the level of ontologies, which can be then propagated to instance knowledge graph, too. While ontology transformation based on symbolic patterns has been around as task for over a decade, its wider adoption was hindered by the overwhelming demand for manual tuning of its individual steps. Nowadays, with the increasing power of pre-trained large language models (LLMs), this hindrance could however be lifted. Given the large number of available LLMs as well as the variety of strategies for their prompting, the developers of ontology transformation architectures need a user-friendly tool for experimenting with different settings. The presented demo provides form fields for input entity names, allows to alter LLMs and parameters, and provides a simple diagrammatic display of the models’ output within the processed ontology structure. Two ontology transformation patterns are currently supported.