This chapter focuses on prompt engineering, a crucial technique for effectively using large language models (LLMs) to solve practical tasks. It explores various strategies for designing prompts, including manual and automatic prompt optimization. Manual prompt design involves four key elements: task description, input data, context information, and prompt strategy, ensuring that prompts are clear, structured, and effective. Additionally, automatic prompt optimization is explored through discrete and continuous methods, including reinforcement learning, gradient-based approaches, and LLM-generated prompts. Strategies such as in-context learning (ICL), which incorporates task examples into prompts, and chain-of-thought (CoT) prompting, which guides models through step-by-step reasoning, are discussed in detail. Advanced strategies such as tree-of-thought (ToT) and graph-of-thought (GoT) extend CoT by introducing structured reasoning frameworks to improve complex problem-solving. The chapter highlights the importance of demonstration design, formatting, ordering, and model-friendly prompts in maximizing LLM performance. Finally, it also examines retrieval-augmented generation (RAG), which enhances prompt responses by incorporating external information. In general, this chapter provides a comprehensive overview of the fundamental approaches to leveraging LLMs via prompting.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Prompt Engineering

  • Wayne Xin Zhao,
  • Kun Zhou,
  • Junyi Li,
  • Tianyi Tang,
  • Ji-Rong Wen

摘要

This chapter focuses on prompt engineering, a crucial technique for effectively using large language models (LLMs) to solve practical tasks. It explores various strategies for designing prompts, including manual and automatic prompt optimization. Manual prompt design involves four key elements: task description, input data, context information, and prompt strategy, ensuring that prompts are clear, structured, and effective. Additionally, automatic prompt optimization is explored through discrete and continuous methods, including reinforcement learning, gradient-based approaches, and LLM-generated prompts. Strategies such as in-context learning (ICL), which incorporates task examples into prompts, and chain-of-thought (CoT) prompting, which guides models through step-by-step reasoning, are discussed in detail. Advanced strategies such as tree-of-thought (ToT) and graph-of-thought (GoT) extend CoT by introducing structured reasoning frameworks to improve complex problem-solving. The chapter highlights the importance of demonstration design, formatting, ordering, and model-friendly prompts in maximizing LLM performance. Finally, it also examines retrieval-augmented generation (RAG), which enhances prompt responses by incorporating external information. In general, this chapter provides a comprehensive overview of the fundamental approaches to leveraging LLMs via prompting.