The rapid advancements in large language models (LLMs) have significantly influenced various natural language processing tasks, yet the detection of metaphorical language remains a challenging area. This study investigates the capabilities of prominent LLMs, specifically OpenAI’s GPT and Google’s Gemini, in identifying metaphorical sentences within Turkish texts. Initially, a dataset comprising metaphorical and literal sentences was curated from diverse sources, including literary works and daily communications, to enrich the linguistic context. Subsequently, this dataset was annotated independently by two human experts and the aforementioned LLMs, employing a zero-shot classification approach. The reliability of these annotations was quantified using Cohen’s Kappa, revealing a high agreement among human annotators (κ = 0.986) but considerably lower consistency between the LLMs (κ = 0.123). The dataset generated through the labeling process was further evaluated using machine learning models. The findings highlight the current limitations of LLMs in processing metaphorical language, underscoring the need for enhanced model training and algorithm refinement. This study not only contributes to the understanding of LLMs’ performance in metaphor detection but also suggests pathways for future enhancements to achieve more nuanced language comprehension in AI systems. The broader implications of these findings suggest potential improvements in applications ranging from machine translation to semantic analysis.