Large Language Model and Text Generation

Large language models (LLMs) are transformer-based neural network architectures pre-trained with language models as loss functions using extensive amounts of text data, often hundreds of billion or even trillions of words, and a large number of model parameters, often billions. LLMs almost unified all major clinical natural language processing (NLP) tasks and aspire to be the one and only solution in NLP. This chapter presents an introduction to LLMs and current progress in biomedical NLP. We introduce the mainstream LLMs including the encoder-only LLMs, decoder-only LLMs, and encoder-decoder LLMs, and discuss their differences. We introduce pre-trained LLMs widely used in the general domain, biomedical domain, and clinical domain. Then, we focus on prompt-based learning, the key technology to adopt LLMs for various downstream applications. We introduce the basic concepts of prompts and various types of prompts such as cloze prompts, prefix prompts, hard/discrete prompts, and soft/continuous prompts. We also introduce prompt-based learning methods using encoder-only LLMs and decoder-only LLMs. Next, we focus on the ability of generative LLMs for synthetic text generation and introduce widely used generation algorithms and evaluation metrics.