Pre-trained language models are integral to the field of natural language processing, continuously evolving as a prominent area of research. This chapter first introduces the concept and development history of pre-trained language models; then, it details the structure, features, and advantages of standard auto-encoding models and autoregressive models such as BERT, GPT-3, LSTM-based ELMo, and ERNIE, as well as the use of pre-trained language models; finally, it provides an outlook and analysis of the development trends and prospects of pre-trained language models.