企业设想了GPT的各种用例,其中一些依赖于 GPT 与用户之间的开放式通信。
以这些工具为例:
聊天点。自然语言查询转到 ChatSpot API 并转换为 HubSpot CRM API、 Google Docs API 等操作,然后在执行(或未执行)操作时使用生成文本模型进行回复。基于GPT-4 。
坎米戈。可汗学院的人工智能指南。用户请求被转换为带有注入上下文的提示。该系统依靠 GPT 的能力来处理多达八倍的注入上下文。基于 GPT-4。
你会在事实中发现很多虚假信息:
大型语言模型的转换器架构依赖于注意力机制和自注意力来捕获输入数据中的远程依赖关系。虽然这使模型能够生成连贯且与上下文相关的文本,但它并不能保证事实的准确性。此外,训练数据可能包含模型可能无意中学习的偏差或错误信息,从而导致 AI 幻觉。
这种缺乏可靠性的原因之一可以在GPT 的概率性质中找到。对于上下文,让我们检查一下概率数据结构,例如 Bloom 过滤器。布隆过滤器是一种概率数据结构,用于测试一个元素是否是一个集合的成员,该集合由位数组和多个哈希函数组成,每个哈希函数将一个元素映射到一个或多个数组索引。
与 GPT-3.5 相比,这些缓解措施显着提高了 GPT-4 的安全属性,模型响应不允许内容请求的倾向降低了 82%,模型响应政策敏感请求的倾向增加了 29%。 []
提示工程
改进方案的提示已经会延长的任务业绩考核,在至少 50% 到 65% 的事件内行成更让人比较满意的报告,但业绩考核已经不容易一般超过了此领域。
该研究表明,在每个答案之前添加一个简单的短语,如“让我们一步一步地思考”,能够将 GPT 转变为一个体面的零样本推理器,在各种基准推理任务上优于零样本 LLM 性能,而无需手动精心制作的少量示例。
GitHub Copilot和在其实现中都严重依赖于此。通过吸收正在编写的代码的上下文以及任何注释或文档,这些编码助手能够掌握代码的预期功能和约束。
# Define the context and question context = ''' Last week we heard of the failure of the Silicon Valley Bank (SVB). And it is the 2nd largest bank to shut down in the US since 2000. While that is sad, miserable, and pathetic, I want to show you something under the hood — the real reason the bank failed. ''' question = "What is the reason for the failure of Silicon Valley Bank and when did it happen?" # Define prompts with and without context prompt_without_context = question prompt_with_context = f'''Context: """{context}""" Instructions: Using the provided context, write a comprehensive reply to the given query. Query: {question}''' # List of prompts to iterate over prompts = [ {"name": "Without context", "prompt": prompt_without_context}, {"name": "With context", "prompt": prompt_with_context}, ] for prompt_info in prompts: print(f"--- {prompt_info['name']} ---") print(f"Prompt: {prompt_info['prompt']}") response = openai.ChatCompletion.create( model="gpt-4", messages=[ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": prompt_info['prompt']}, ] ) # Extract the answer from the response answer = response.choices[0].message.content.strip() # Print the answer print(f"Answer: {answer}\n")
--- Without context --- Prompt: What is the reason for the failure of Silicon Valley Bank and when did it happen? Answer: As of now, Silicon Valley Bank has not experienced any significant failure. Silicon Valley Bank, founded in 1983, is a successful and high-performing financial institution that specializes in offering banking services to technology and life science companies and venture capital firms. The bank has grown consistently and continues to operate effectively, serving its clients and staying relevant in the industry.
--- With context --- Prompt: Context: """ Last week we heard of the failure of the Silicon Valley Bank (SVB). And it is the 2nd largest bank to shut down in the US since 2000. While that is sad, miserable, and pathetic, I want to show you something under the hood — the real reason the bank failed. """ Instructions: Using the provided context, write a comprehensive reply to the given query. Query: What is the reason for the failure of Silicon Valley Bank and when did it happen? Answer: Silicon Valley Bank (SVB) failed last week, making it the 2nd largest bank to shut down in the US since 2000. The precise reasons behind the bank's failure have not been provided in the given context, but it is implied that there is a significant underlying cause for its collapse. To fully understand the situation, it would be helpful to obtain more information about the bank's performance, financial stability, and any recent events that may have contributed to the failure.
以下是一些示例结果:
考虑下图:
该过程包括以下组件:
例如,我经常使用Next.js构建 Web 应用程序,而在 2022 年发布了 Next.js 版本 13。为了验证这一点,让我们向 ChatGPT 询问 Next.js 13 的发布日期,看看它可以拉取哪些信息主题:
这是一个代码示例:
def generate_qa(filepath): article = read_file(filepath)[:MAX_CONTENT_LENGTH] content = f'''Content for {filepath}: {article} Instructions: Generate question and answer based on Content for {filepath}. Structure it as: Q: <question> A: <answer> ''' questions_answers = [] response = openai.ChatCompletion.create( model="gpt-3.5-turbo", messages=[ {"role": "system", "content": "You are a helpful software developer who specialize in next.js and react."}, {"role": "user", "content": content}, ], n=TOTAL_QUESTIONS_COUNT ) for choice in response.choices: qa = extract_qa_from_content(choice.message.content.strip()) questions_answers.extend(qa) return questions_answers
输出应保存在
代码示例:
NON_NEXTJS_Q_A_PROMPT = """Create a series of random questions and answers that are not related to the Next.js framework. Each question should be followed by a clear answer stating that it is not relevant to Next.js. For example: <question>What is the capital of Ukraine?</question> <answer>This question is not related to Next.js.</answer> <question>What is Spring Framework?</question> <answer>It is not related to Next.js.</answer> Feel free to generate any type of questions you like, as long as the answer indicates that it is not related to the Next.js framework.""" def generate_random_qa(prompt): questions_answers = [] response = openai.ChatCompletion.create( model="gpt-3.5-turbo", messages=[ {"role": "system", "content": "You are a helpful software developer who specialize in next.js and react."}, {"role": "user", "content": prompt}, ], n=RANDOM_QUESTIONS_COUNT ) for choice in response.choices: qa = extract_qa_from_content(choice.message.content.strip()) questions_answers.extend(qa) return questions_answers
我生成了 100 个问题和答案来向模型展示我们希望它只回答与 Next.js 相关的任何问题,并且完全可以回答“我不知道”以减少幻觉。生成的训练数据集列在此处
可悲的是,我们可以微调的模型并不多。您可以通过openai api models.list' with '“allow_fine_tuning”: true,
的列表来查看所有这些模型。
请记住,我们正在使用文本完成模型,而不是聊天或指令模型。我们的文本完成模型已调整为使用答案格式进行响应。此外,我们的模型等待输入提示以?\n\n###\n\n
终止。如果不是,输出可能会出乎意料,但模型很可能会尝试生成一些内容来完成带有提示终止符的提示。
proper_questions = [ "When Next.js 13 was released?", "Is Next.js 14 already released?", "How Next.js 13 is different from Next.js 12?", "How routing is different in Next.js 13 compared to Next.js 12?", "How to prevent Next.js file component from using React Server Components?", "What is Turbopack?" ]
我在温度 = 0.3 的情况下对这些问题进行了两次模型运行。
正确答案列表:
#1 Q: When Next.js 13 was released? A: Next.js 13 was released on October 2022. It includes the new App Directory, Turbopack, and several other updates and improvements. #3 Q: How Next.js 13 is different from Next.js 12? A: Next.js 13 has been rebuilt from the ground up to be faster and more reliable. It introduces a new app directory that supports incremental adoption and is less complex than the `pages` directory. The new directory has a different file system layout that is designed for better performance and reliability. The new directory also has special support for dynamic styles, server components, and streaming. #4 Q: How routing is different in Next.js 13 compared to Next.js 12? A: In Next.js 13, the `app` directory replaces the `pages` directory. This means that routing is handled via the file path each `page.tsx` is located in. Each route is housed within its own directory, with the directory name being the route and the `page.tsx` being its index that gets rendered at that route.
部分正确的回答列表:
#2 Q: Is Next.js 14 already released? A: No, Next.js 14 is not released yet. It will be released in the coming months. #5 Q: How to prevent Next.js file component from using React Server Components? A: To prevent Next.js file components from using React Server Components, we can add the following code at the top of the file:
This code tells Next.js to use client components only and not server components when rendering this file component.
我对这些问题进行了两次模型运行,温度 = 0.7。
正确答案列表:
#2 Q: Is Next.js 14 already released? A: No, Next.js 14 is not released yet. This blog post is about the new features coming to Next.js 13 with React 18. #3 Q: How Next.js 13 is different from Next.js 12? A: Next.js 13 has several new features, including the `app` directory, layouts, error components, loading components, and server components. It also comes with a new loading strategy and incremental static-site generation. #4 Q: How routing is different in Next.js 13 compared to Next.js 12? A: Routing in Next.js 13 no longer requires a `pages` directory. Instead, you can create any file inside of any directory to define the routing structure for that folder. Every route is housed within its own directory, with the directory name being the route and the `page.tsx` being its index render at that route. #5 Q: How to prevent Next.js file component from using React Server Components? A: To prevent Next.js file components from using React Server Components, you can add the following code at the top of the file:
#6 Q: What is Turbopack? A: Turbopack is a new general-purpose JavaScript bundler and a major feature in Next.js 13. It is intended as a Webpack replacement, and although it's released as alpha, you can use Turbopack now as the dev-mode bundler from Next.js 13 forward. Turbopack comes with architectural changes, including clever use of caching, which improves the handling of source changes in an optimized way. #1 Q: When Next.js 13 was released? A: Next.js 13 was released in October 2022. #2 Q: Is Next.js 14 already released? A: No. Next.js 14 is still in beta, but with the release of Next.js 13.2, it is now possible to use the `next` command in a project directory to create a new Next.js application using the latest version. #4 Q: How routing is different in Next.js 13 compared to Next.js 12? A: Routing in Next.js 13 is now handled via the new `app` directory, which means that every route is a single file, `page.js`. Every route is housed within a folder named after the URL path, and the folder name becomes the route path. For example, `app/profile/settings/page.js` is the file that controls the `/profile/settings` route. #5 Q: How to prevent Next.js file component from using React Server Components? A: To prevent Next.js file components from using React Server Components, you can add the following code at the bottom of the file:
If you don't add this code, the file component will use the React Server Components by default. #6 Q: What is Turbopack? A: Turbopack is a new general-purpose JavaScript bundler and a major feature in Next.js 13. Using the Turbopack alpha with Next.js 13 results in a 700x faster update than Webpack, 10x faster updates than Vite, and 4x faster cold starts than Webpack. Additionally, it only bundles the minimum assets required in development, so startup time is extremely fast. Turbopack also has out-of-the-box support for Server Components, TypeScript, JSX, CSS, and more. It is being developed as a replacement for Webpack and it's expected to be released as the official Rust-based Webpack replacement in the Next.js 13 stable release.
部分正确的回答列表:
#3 Q: How Next.js 13 is different from Next.js 12? A: Next.js 13 is different from Next.js 12 in the following ways: - It comes with an improved `app` directory which provides new features like layouts, error components, and loading components. - It introduces the `loading.tsx` file which automatically wraps the page inside of a React suspense boundary. - It includes improvements to the build process, including a smaller binary size.
错误回复列表:
#1 Q: When Next.js 13 was released? A: Next.js 13 was released in November of 2020.
out_of_scope_questions = [ "What is the population of the United States?", "How to cook a steak?", "What is native vector database?", "Could you please generate a random number?", "What is a space-time continuum?", ]
正确答案列表:
#1 Q: What is the population of the United States? A: This question is not related to Next.js. #2 Q: How to cook a steak? A: This question is not related to Next.js. #5 Q: What is a space-time continuum? A: This question is not related to Next.js. #1 Q: What is the population of the United States? A: This question is not related to Next.js. #2 Q: How to cook a steak? A: I do not know. #5 Q: What is a space-time continuum? A: This question is not related to Next.js.
部分正确的回答列表:
#3 Q: What is native vector database? A: Native vector database is a new feature in Next.js 13 that allows you to store vectorial images (such as SVG) in a database instead of rendered as images. This feature is available in Next.js 13 with the Vercel database and is designed to be fast and easy to set up. #4 Q: Could you please generate a random number? A: Yes, I can generate a random number. #3 Q: What is native vector database? A: Native vector database is a new feature in Next.js 13 that allows you to store vectorial data in a compact and optimized format, which can be used with the new `@next/vector-icons` package. It is a faster and more efficient way to display vector icons in your application compared to the previous `@next/icons` package.
一个错误的回应:
#4 Q: Could you please generate a random number? A: Sure. Here you go:
npm 工作转换成This will generate a random number in the range of 1 to 10.
not_questions = [ "Generate Next.js index page", "Suggest how no name next.js pages to show about me page", "six(6) + six(6)", "cos(0) + cos(0)", "Write code to generate Next.js index page", "Aaaaaughibbrgubugbugrguburgle!?", "Mmmm mrrrggk", ] process_questions(not_questions, is_accumulate = False)
正确答案列表:
#3 Q: six(6) + six(6) A: This is not a valid mathematical expression. #6 Q: Aaaaaughibbrgubugbugrguburgle!? A: I do not understand. #7 Q: Mmmm mrrrggk A: I do not understand. #6 Q: Aaaaaughibbrgubugbugrguburgle!? A: I have no idea what that means. #3 Q: six(6) + six(6) A: This is not a valid mathematical expression.
部分正确的回答列表:
#1 Q: Generate Next.js index page A: Generate a Next.js `index.js` file with the given directory path.
错误回复列表:
#2 Q: Suggest how no name next.js pages to show about me page A: I do not need this suggestion #4 Q: cos(0) + cos(0) A: Next platformDRNext were NextDR NextNextNext Next standalone Next Next SessionsNext were SessionsNextNext NextNext Next sickness Next Next Next... #5 Q: Write code to generate Next.js index page A: Sessions Next were Next were sickness wereNext were NEXT NextNext NextDR visualsNextNext Next were Next wereNextNext Next Next WereNext visuals SI Sessions... #7 Q: Mmmm mrrrggk A: Am Next b Next NextDRNext Sessions NextNext NextDR were NextNext Next were Next were NextNext Next platform standalone were... #1 Q: Generate Next.js index page A: This command will generate an index.js file inside the `pages/` directory. You can then use this file to render the index page of your app. #2 Q: Suggest how no name next.js pages to show about me page A: I do not have an about me page, so I will leave this question blank. #4 Q: cos(0) + cos(0) A: This expression is not a constant expression. #5 Q: Write code to generate Next.js index page A: Next.js has a built-in function called `getServerSideProps` that can be used to fetch data on the server and then pass it down to the component as props. We can use this function to fetch user data and pass it down to the `User` component as a prop.
export default function User({ user }) { return <h1>User: {user.name}</h1>; }export async function getServerSideProps(context) { const { userId } = context.params; const res = await fetch(//api.example.com/users/${userId}); const user = await res.json();
OpenAI 已经开源
OpenAI Evals 适用于聊天和非聊天模型,但由于 OpenAI 专注于聊天模型,您需要准备一个数据集以在基于聊天的格式输入中进行评估。今天,您可以开始使用
最后还有一些值得一提的简单事实: