Large Language Models (LLMs) have become increasingly popular in academia and industry due to their ability to comprehend natural language and generate content to perform various tasks. Numerous studies suggested that service-oriented computing (SOC), a key computing paradigm for building distributed enterprise applications, can benefit from LLMs. We observed that SOC researchers are increasingly applying LLMs to tasks such as service composition, service code generation, and service monitoring. However, no study systematically reviews these research efforts and delineates research challenges. To fill this research gap, this study presents a systematic literature review of 64 papers published on LLMs for SOC tasks between January 2019 and February 2025. We aimed to analyze and summarize the relevant studies and formulate a research roadmap to provide a context for shaping ongoing research activities. We categorized our findings into four SOC domains: service foundation, service composition, monitoring and management, and design and development.