The increasing complexity and size of large language models (LLMs) have necessitated the development of sophisticated parallel strategies to optimize their training and deployment. To enhance the utilization of computational resources, parallelization techniques are extensively studied and implemented. However, current parallel strategies cannot be dynamically adjusted after their initialization. In this paper, we review the literature on parallel strategies for LLMs in both training and inference scenarios, emphasizing the need for adaptable parallel strategies. We first introduce the five primary parallel strategies: data parallelism, tensor parallelism, pipeline parallelism, sequence parallelism, and expert parallelism. Next, we present their applications for LLMs and explore the growing demand for adaptable parallel strategies. Finally, we discuss the challenges associated with these strategies and outline directions for future research.