Large language models (LLMs) have demonstrated remarkable performance in various natural language processing tasks. However, their abilities in generating specific syntactic functions still need to be tested. Using verbs of chatting as a case study, this paper applied dependency grammar to annotate and extract their syntactic information. On this basis, prompts were utilized to guide LLMs in generating sentences of sentence-head relations and verb-object relations. The syntactic ability of the LLMs was then evaluated through error analysis, providing insights into the models’ performance at the syntactic level. The results reveal limitations in the LLMs’ ability to generate sentences with specific syntactic functions.