Prompt Engineering

Prompt Engineering Prompt Engineering Modified December 31, 2024 External APIs 外部 API TALM (Tool Augmented Language Models; Parisi et al. 2022 ) is a language model augmented with text to text API calls. LM is guided to generate |tool call and tool input text conditioned on task input text to construct API call requests. When |result shows up, the specified tool API is called and the returned result gets appended to the text sequence. The final output is generated following |output token. TALM（工具增强语言模型；Parisi 等人，2022）是一种通过文本到文本 API 调用增强的语言模型。LM 被引导生成 |tool call 和 tool input text ，根据任务输入文本构建 API 调用请求。当 |result 出现时，调用指定的工具 API，并将返回的结果附加到文本序列中。最终输出在 |output 标记后生成。 Fig. 4. The format of API calls in TALM. (Image source: Parisi et al. 2022 ). 图 4. TALM 中 API 调用的格式。（图片来源：Parisi 等，2022）。 TALM adopts a self play approach to iteratively bootstrap the dataset of tool use examples and finetune LM with it. TALM 采用自我对弈的方法，迭代地自举工具使用示例数据集，并利用其微调语言模型。 This self play, defined as a model interacting with a tool API, iteratively expands the dataset based on whether a newly added tool API can improve the model outputs. Same idea is adopted in Toolformer too, described in more details below. 这种自我对弈，定义为模型与工具 API 的交互，根据新添加的工具 API 是否能改进模型输出来迭代扩展数据集。Toolformer 也采用了相同的理念，下面将更详细地描述。 The pipeline loosely mimics a RL process where LM is the policy network and it is trained by policy gradient with a binary reward signal. 该管道松散地模仿了一个强化学习过程，其中语言模型作为策略网络，并通过带有二元奖励信号的策略梯度进行训练。 Fig. 5. Self play iterations help boost the model performance. 图 5. 自我对弈迭代有助于提升模型性能。 (Image source: Parisi et al. 2022 ). （图片来源：Parisi 等人，2022 年）。 Toolformer ( Schick et al. 2023 ) is a LM that can use external tools via simple APIs, which is built in a self supervised manner and only requires a handful of demonstrations for each API. The toolbox of Toolformer includes: Toolformer（Schick 等人，2023）是一种可以通过简单 API 使用外部工具的语言模型，它以自监督的方式构建，每个 API 仅需少量演示。Toolformer 的工具箱包括： • Calculator to help LM with the lack of precise math skills; 计算器用于帮助 LM 弥补精确数学技能的不足； • Q&A system to help with unfaithful content and hallucination; 问答系统，用于处理不忠实内容和幻觉问题； • Search engine to provide up to date information after pretraining cut off time; 搜索引擎在预训练截止时间后提供最新信息； • Translation system to improve performance on low resource language; 翻译系统以提高低资源语言的性能 • Calendar to make LM be aware of time progression. 日历用于让 LM 意识到时间的流逝。 Fig. 6. Illustration of how to build Toolformer. 图 6. 如何构建 Toolformer 的示意图。 (Image source: Schick et al. 2023 ). （图片来源：Schick 等人，2023 年）。 Toolformer is trained as follows: Toolformer 的训练过程如下： 1. Prompting to annotate potential API calls . Ask a pre trained LM to annotate a dataset via few shot learning with API call usage examples. Formatting example: 提示标注潜在的 API 调用。通过少量示例学习，要求预训练语言模型标注包含 API 调用使用的数据集。格式化示例： 1. Fig. 7. How dataset is annotated to do API calls. 图 7. 数据集如何被标注以进行 API 调用。 (Image source: Schick et al. 2023 ). （图片来源：Schick 等人，2023）。 ◦ Each API call is represented as a tuple of (API name, corresponding input), and its corresponding result is denoted as . The API call sequences with and without results are labeled as follows, respectively: 每个 API 调用表示为一个（API 名称，对应输入）的元组， 其对应结果表示为 。带有结果和不带结果的 API 调用序列分别标记如下： ◦ Sample API calls based on the probabilities and select top candidate positions for doing API calls at position if the probability is larger than a threshold. 基于概率 的示例 API 调用，并在位置 选择前 个候选位置进行 API 调用，如果概率大于阈值。 ◦ Then we sample potential API calls from the LM given the sequence as prefix and as suffix. 然后我们从语言模型中采样潜在的 API 调用，给定序列 作为前缀， 作为后缀。 2. Filter annotations based on whether API calls help model predict future tokens. Use a self supervised loss to decide which API calls are actually helpful. 根据 API 调用是否有助于模型预测未来标记来过滤注释。使用自监督损失来决定哪些 API 调用实际上是有帮助的。 ◦ Execute each API call to get corresponding result . 执行每个 API 调用 以获取相应的结果 。 ◦ Compute weighted cross entropy loss for the LM over tokens when the model is prefixed with the prompt. Two versions are computed, one with API result and the other with empty sequence . 计算语言模型在带有提示前缀时对标记 的加权交叉熵损失。计算两个版本，一个使用 API 结果，另一个使用空序列 。 ◦ Only API calls with larger than a threshold are kept, meaning that adding this API call and its results help the model predict future tokens. 仅保留 大于阈值的 API 调用，这意味着添加此 API 调用及其结果有助于模型预测未来的标记。 3. Fine tune LM on this annotated dataset. The new training sequences are constructed as . The training data is a combination of the original dataset (e.g. a subset of CCNet, as in the paper) and its augmented version. 在此标注数据集上微调语言模型。新的训练序列构建为 。训练数据是原始数据集（例如论文中的 CCNet 子集）及其增强版本的组合。 At inference time, decoding runs until the model produces “ " token, indicating that it is expecting response from an API call next. 在推理时，解码过程将持续进行，直到模型生成“ ”标记，表明它接下来期待从 API 调用中获得响应。 Toolformer currently does not support tool use in a chain (i.e. using the output of one tool as an input for another tool) or in an interactive way (i.e. adopt API response after human selection). Both are interesting future directions to expand the model for. Toolformer 目前不支持链式工具使用（即使用一个工具的输出作为另一个工具的输入）或交互式使用（即在人工选择后采用 API 响应）。这两者都是未来扩展模型的有趣方向。 Citation 引用 Cited as: 引用为： Weng, Lilian. (Mar 2023). Prompt Engineering. Lil’Log. https://lilianweng.github.io/posts/2023 03 15 prompt engineering/. Weng, Lilian. (2023 年 3 月). 提示工程. Lil’Log. https://lilianweng.github.io/posts/2023 03 15 prompt engineering/. Or 或 Code block CoffeeScript Copy @article{weng2023prompt, title = "Prompt Engineering", author = "Weng, Lilian", journal = "lilianweng.github.io", year = "2023", month = "Mar", url = "https://lilianweng.github.io/posts/2023 03 15 prompt engineering/" } Useful Resources 有用资源 • OpenAI Cookbook has many in depth examples for how to utilize LLM efficiently. OpenAI Cookbook 提供了许多深入示例，展示了如何高效利用 LLM。 • LangChain , a library for combining language models with other components to build applications. LangChain，一个将语言模型与其他组件结合以构建应用程序的库。 • Prompt Engineering Guide repo contains a pretty comprehensive collection of education materials on prompt engineering. Prompt Engineering Guide 仓库包含了一套相当全面的关于提示工程的教育材料。 • learnprompting.org • PromptPerfect • Semantic Kernel 语义内核 References 参考文献 [1] Zhao et al. “Calibrate Before Use: Improving Few shot Performance of Language Models.” ICML 2021 [1] 赵等人。“使用前校准：提升语言模型的少样本性能。”ICML 2021 [2] Liu et al. “What Makes Good In Context Examples for GPT 3?” arXiv preprint arXiv:2101.06804 (2021). [2] 刘等人. “什么构成了 GPT 3 的良好上下文示例？” arXiv 预印本 arXiv:2101.06804 (2021). [3] Lu et al. “Fantastically Ordered Prompts and Where to Find Them: Overcoming Few Shot Prompt Order Sensitivity.” ACL 2022 [3] 卢等人。“神奇有序的提示及其发现之道：克服少样本提示顺序敏感性。”ACL 2022 [4] Ye et al. “In Context Instruction Learning.” arXiv preprint arXiv:2302.14691 (2023). [4] 叶等人. “上下文指令学习.” arXiv 预印本 arXiv:2302.14691 (2023). [5] Su et al. “Selective annotation makes language models better few shot learners.” arXiv preprint arXiv:2209.01975 (2022). [5] Su 等。“选择性注释使语言模型成为更好的少样本学习者。”arXiv 预印本 arXiv:2209.01975（2022）。 [6] Rubin et al. “Learning to retrieve prompts for in context learning.” NAACL HLT 2022 [6] Rubin 等人，“学习检索提示以进行上下文学习。” NAACL HLT 2022 [7] Wei et al. “Chain of thought prompting elicits reasoning in large language models.” NeurIPS 2022 [7] Wei 等人。“思维链提示在大型语言模型中引发推理。”NeurIPS 2022 [8] Wang et al. “Self Consistency Improves Chain of Thought Reasoning in Language Models.” ICLR 2023. [8] 王等人。“自一致性提升语言模型中的思维链推理。”ICLR 2023。 Parisi et al. 2022 Parisi et al. 2022 Parisi et al. 2022 Schick et al. 2023 Schick et al. 2023 Schick et al. 2023 OpenAI Cookbook LangChain Prompt Engineering Guide learnprompting.org PromptPerfect Semantic Kernel 语义内核 “Calibrate Before Use: Improving Few shot Performance of Language Models.” “What Makes Good In Context Examples for GPT 3?” “Fantastically Ordered Prompts and Where to Find Them: Overcoming Few Shot Prompt Order Sensitivity.” “In Context Instruction Learning.” “Selective annotation makes language models better few shot learners.” “Learning to retrieve prompts for in context learning.” “Chain of thought prompting elicits reasoning in large language models.” “Self Consistency Improves Chain of Thought Reasoning in Language Models.” External APIs 外部 API TALM (Tool Augmented Language Models; Parisi et al. 2022 ) is a language model augmented with text to text API calls. LM is guided to generate |tool call and tool input text conditioned on task input text to construct API call requests. When |result shows up, the specified tool API is called and the returned result gets appended to the text sequence. The final output is generated following |output token. TALM（工具增强语言模型；Parisi 等人，2022）是一种通过文本到文本 API 调用增强的语言模型。LM 被引导生成 |tool call 和 tool input text ，根据任务输入文本构建 API 调用请求。当 |result 出现时，调用指定的工具 API，并将返回的结果附加到文本序列中。最终输出在 |output 标记后生成。 Parisi et al. 2022 Fig. 4. The format of API calls in TALM. (Image source: Parisi et al. 2022 ). 图 4. TALM 中 API 调用的格式。（图片来源：Parisi 等，2022）。 Parisi et al. 2022 TALM adopts a self play approach to iteratively bootstrap the dataset of tool use examples and finetune LM with it. TALM 采用自我对弈的方法，迭代地自举工具使用示例数据集，并利用其微调语言模型。 This self play, defined as a model interacting with a tool API, iteratively expands the dataset based on whether a newly added tool API can improve the model outputs. Same idea is adopted in Toolformer too, described in more details below. 这种自我对弈，定义为模型与工具 API 的交互，根据新添加的工具 API 是否能改进模型输出来迭代扩展数据集。Toolformer 也采用了相同的理念，下面将更详细地描述。 The pipeline loosely mimics a RL process where LM is the policy network and it is trained by policy gradient with a binary reward signal. 该管道松散地模仿了一个强化学习过程，其中语言模型作为策略网络，并通过带有二元奖励信号的策略梯度进行训练。 Fig. 5. Self play iterations help boost the model performance. 图 5. 自我对弈迭代有助于提升模型性能。 (Image source: Parisi et al. 2022 ). （图片来源：Parisi 等人，2022 年）。 Parisi et al. 2022 Toolformer ( Schick et al. 2023 ) is a LM that can use external tools via simple APIs, which is built in a self supervised manner and only requires a handful of demonstrations for each API. The toolbox of Toolformer includes: Toolformer（Schick 等人，2023）是一种可以通过简单 API 使用外部工具的语言模型，它以自监督的方式构建，每个 API 仅需少量演示。Toolformer 的工具箱包括： Schick et al. 2023 • Calculator to help LM with the lack of precise math skills; 计算器用于帮助 LM 弥补精确数学技能的不足； • Q&A system to help with unfaithful content and hallucination; 问答系统，用于处理不忠实内容和幻觉问题； • Search engine to provide up to date information after pretraining cut off time; 搜索引擎在预训练截止时间后提供最新信息； • Translation system to improve performance on low resource language; 翻译系统以提高低资源语言的性能 • Calendar to make LM be aware of time progression. 日历用于让 LM 意识到时间的流逝。 Fig. 6. Illustration of how to build Toolformer. 图 6. 如何构建 Toolformer 的示意图。 (Image source: Schick et al. 2023 ). （图片来源：Schick 等人，2023 年）。 Schick et al. 2023 Toolformer is trained as follows: Toolformer 的训练过程如下： 1. Prompting to annotate potential API calls . Ask a pre trained LM to annotate a dataset via few shot learning with API call usage examples. Formatting example: 提示标注潜在的 API 调用。通过少量示例学习，要求预训练语言模型标注包含 API 调用使用的数据集。格式化示例： 1. Fig. 7. How dataset is annotated to do API calls. 图 7. 数据集如何被标注以进行 API 调用。 (Image source: Schick et al. 2023 ). （图片来源：Schick 等人，2023）。 ◦ Each API call is represented as a tuple of (API name, corresponding input), and its corresponding result is denoted as . The API call sequences with and without results are labeled as follows, respectively: 每个 API 调用表示为一个（API 名称，对应输入）的元组， 其对应结果表示为 。带有结果和不带结果的 API 调用序列分别标记如下： ◦ Sample API calls based on the probabilities and select top candidate positions for doing API calls at position if the probability is larger than a threshold. 基于概率 的示例 API 调用，并在位置 选择前 个候选位置进行 API 调用，如果概率大于阈值。 ◦ Then we sample potential API calls from the LM given the sequence as prefix and as suffix. 然后我们从语言模型中采样潜在的 API 调用，给定序列 作为前缀， 作为后缀。 Schick et al. 2023 ◦ Each API call is represented as a tuple of (API name, corresponding input), and its corresponding result is denoted as . The API call sequences with and without results are labeled as follows, respectively: 每个 API 调用表示为一个（API 名称，对应输入）的元组， 其对应结果表示为 。带有结果和不带结果的 API 调用序列分别标记如下： ◦ Sample API calls based on the probabilities and select top candidate positions for doing API calls at position if the probability is larger than a threshold. 基于概率 的示例 API 调用，并在位置 选择前 个候选位置进行 API 调用，如果概率大于阈值。 ◦ Then we sample potential API calls from the LM given the sequence as prefix and as suffix. 然后我们从语言模型中采样潜在的 API 调用，给定序列 作为前缀， 作为后缀。 2. Filter annotations based on whether API calls help model predict future tokens. Use a self supervised loss to decide which API calls are actually helpful. 根据 API 调用是否有助于模型预测未来标记来过滤注释。使用自监督损失来决定哪些 API 调用实际上是有帮助的。 ◦ Execute each API call to get corresponding result . 执行每个 API 调用 以获取相应的结果 。 ◦ Compute weighted cross entropy loss for the LM over tokens when the model is prefixed with the prompt. Two versions are computed, one with API result and the other with empty sequence . 计算语言模型在带有提示前缀时对标记 的加权交叉熵损失。计算两个版本，一个使用 API 结果，另一个使用空序列 。 ◦ Only API calls with larger than a threshold are kept, meaning that adding this API call and its results help the model predict future tokens. 仅保留 大于阈值的 API 调用，这意味着添加此 API 调用及其结果有助于模型预测未来的标记。 ◦ Execute each API call to get corresponding result . 执行每个 API 调用 以获取相应的结果 。 ◦ Compute weighted cross entropy loss for the LM over tokens when the model is prefixed with the prompt. Two versions are computed, one with API result and the other with empty sequence . 计算语言模型在带有提示前缀时对标记 的加权交叉熵损失。计算两个版本，一个使用 API 结果，另一个使用空序列 。 ◦ Only API calls with larger than a threshold are kept, meaning that adding this API call and its results help the model predict future tokens. 仅保留 大于阈值的 API 调用，这意味着添加此 API 调用及其结果有助于模型预测未来的标记。 3. Fine tune LM on this annotated dataset. The new training sequences are constructed as . The training data is a combination of the original dataset (e.g. a subset of CCNet, as in the paper) and its augmented version. 在此标注数据集上微调语言模型。新的训练序列构建为 。训练数据是原始数据集（例如论文中的 CCNet 子集）及其增强版本的组合。 At inference time, decoding runs until the model produces “ " token, indicating that it is expecting response from an API call next. 在推理时，解码过程将持续进行，直到模型生成“ ”标记，表明它接下来期待从 API 调用中获得响应。 Toolformer currently does not support tool use in a chain (i.e. using the output of one tool as an input for another tool) or in an interactive way (i.e. adopt API response after human selection). Both are interesting future directions to expand the model for. Toolformer 目前不支持链式工具使用（即使用一个工具的输出作为另一个工具的输入）或交互式使用（即在人工选择后采用 API 响应）。这两者都是未来扩展模型的有趣方向。 Citation 引用 Cited as: 引用为： Weng, Lilian. (Mar 2023). Prompt Engineering. Lil’Log. https://lilianweng.github.io/posts/2023 03 15 prompt engineering/. Weng, Lilian. (2023 年 3 月). 提示工程. Lil’Log. https://lilianweng.github.io/posts/2023 03 15 prompt engineering/. Or 或 Useful Resources 有用资源 • OpenAI Cookbook has many in depth examples for how to utilize LLM efficiently. OpenAI Cookbook 提供了许多深入示例，展示了如何高效利用 LLM。 OpenAI Cookbook • LangChain , a library for combining language models with other components to build applications. LangChain，一个将语言模型与其他组件结合以构建应用程序的库。 LangChain • Prompt Engineering Guide repo contains a pretty