This article explores the ability of Large Language Models (LLMs) to semantically annotate RESTful services. In particular, it evaluates how well state-of-the-art LLMs annotate the input and output service parameters with concepts from well-known ontologies. Further, since this evaluation signifies that the annotation based on any LLM does not meet the expected accuracy levels, it presents a sophisticated LLM-based approach that automatically constructs an ontology from the OpenAPI service specifications and exploits that ontology to semantically annotate these specifications. This approach’s evaluation proves that annotation accuracy becomes perfect when using the best-performing LLM.