Skip to content

docs[minor]: Log actual prompt when using prompt hub #5423

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
May 16, 2024
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 33 additions & 5 deletions docs/core_docs/docs/how_to/qa_chat_history_how_to.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,9 @@
"\n",
"In many Q&A applications we want to allow the user to have a back-and-forth conversation, meaning the application needs some sort of \"memory\" of past questions and answers, and some logic for incorporating those into its current thinking.\n",
"\n",
"In this guide we focus on **adding logic for incorporating historical messages, and NOT on chat history management.** Chat history management is [covered here](/docs/expression_language/how_to/message_history).\n",
"In this guide we focus on **adding logic for incorporating historical messages, and NOT on chat history management.** Chat history management is [covered here](/docs/how_to/message_history).\n",
"\n",
"We'll work off of the Q&A app we built over the [LLM Powered Autonomous Agents](https://lilianweng.github.io/posts/2023-06-23-agent/) blog post by Lilian Weng in the [Quickstart](/docs/use_cases/question_answering/quickstart). We'll need to update two things about our existing app:\n",
"We'll work off of the Q&A app we built over the [LLM Powered Autonomous Agents](https://lilianweng.github.io/posts/2023-06-23-agent/) blog post by Lilian Weng. We'll need to update two things about our existing app:\n",
"\n",
"1. **Prompt**: Update our prompt to support historical messages as an input.\n",
"2. **Contextualizing questions**: Add a sub-chain that takes the latest user question and reformulates it in the context of the chat history. This is needed in case the latest question references some context from past messages. For example, if a user asks a follow-up question like \"Can you elaborate on the second point?\", this cannot be understood without the context of the previous message. Therefore we can't effectively perform retrieval with a question like this."
Expand All @@ -23,7 +23,7 @@
"## Setup\n",
"### Dependencies\n",
"\n",
"We’ll use an OpenAI chat model and embeddings and a Memory vector store in this walkthrough, but everything shown here works with any [ChatModel](/docs/concepts/#chat-models) or [LLM](/docs/modules/model_io/llms), [Embeddings](/docs/modules/data_connection/text_embedding/), and [VectorStore](/docs/modules/data_connection/vectorstores/) or [Retriever](/docs/modules/data_connection/retrievers/).\n",
"We’ll use an OpenAI chat model and embeddings and a Memory vector store in this walkthrough, but everything shown here works with any [ChatModel](/docs/concepts/#chat-models) or [LLM](/docs/concepts#llms), [Embeddings](/docs/concepts#embedding-models), and [VectorStore](/docs/concepts#vectorstores) or [Retriever](/docs/concepts#retrievers).\n",
"\n",
"We’ll use the following packages:\n",
"\n",
Expand All @@ -44,7 +44,7 @@
"source": [
"### LangSmith\n",
"\n",
"Many of the applications you build with LangChain will contain multiple steps with multiple invocations of LLM calls. As these applications get more and more complex, it becomes crucial to be able to inspect what exactly is going on inside your chain or agent. The best way to do this is with [LangSmith](https://smith.langchain.com/).\n",
"Many of the applications you build with LangChain will contain multiple steps with multiple invocations of LLM calls. As these applications get more and more complex, it becomes crucial to be able to inspect what exactly is going on inside your chain or agent. The best way to do this is with [LangSmith](/docs/langsmith/).\n",
"\n",
"Note that LangSmith is not needed, but it is helpful. If you do want to use LangSmith, after you sign up at the link above, make sure to set your environment variables to start logging traces:\n",
"\n",
Expand Down Expand Up @@ -81,7 +81,7 @@
},
{
"cell_type": "code",
"execution_count": 5,
"execution_count": 14,
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -99,6 +99,7 @@
"\n",
"// Retrieve and generate using the relevant snippets of the blog.\n",
"const retriever = vectorStore.asRetriever();\n",
"// Tip - you can edit this!\n",
"const prompt = await pull<ChatPromptTemplate>(\"rlm/rag-prompt\");\n",
"const llm = new ChatOpenAI({ model: \"gpt-3.5-turbo\", temperature: 0 });\n",
"const ragChain = await createStuffDocumentsChain({\n",
Expand All @@ -108,6 +109,33 @@
"});"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Lets see what this prompt actually looks like"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"You are an assistant for question-answering tasks. Use the following pieces of retrieved context to answer the question. If you don't know the answer, just say that you don't know. Use three sentences maximum and keep the answer concise.\n",
"Question: {question} \n",
"Context: {context} \n",
"Answer:\n"
]
}
],
"source": [
"console.log(prompt.promptMessages.map((msg) => msg.prompt.template).join(\"\\n\"));"
]
},
{
"cell_type": "code",
"execution_count": 7,
Expand Down
2 changes: 1 addition & 1 deletion docs/core_docs/docs/how_to/qa_citations.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
"## Setup\n",
"### Dependencies\n",
"\n",
"We’ll use an OpenAI chat model and embeddings and a Memory vector store in this walkthrough, but everything shown here works with any [ChatModel](/docs/concepts/#chat-models) or [LLM](/docs/modules/model_io/llms), [Embeddings](https://js.langchain.com/docs/modules/data_connection/text_embedding/), and [VectorStore](https://js.langchain.com/docs/modules/data_connection/vectorstores/) or [Retriever](/docs/modules/data_connection/retrievers/).\n",
"We’ll use an OpenAI chat model and embeddings and a Memory vector store in this walkthrough, but everything shown here works with any [ChatModel](/docs/concepts/#chat-models) or [LLM](/docs/concepts#llms), [Embeddings](https://js.langchain.com/docs/modules/data_connection/text_embedding/), and [VectorStore](https://js.langchain.com/docs/modules/data_connection/vectorstores/) or [Retriever](/docs/concepts#retrievers).\n",
"\n",
"We’ll use the following packages:\n",
"\n",
Expand Down
31 changes: 29 additions & 2 deletions docs/core_docs/docs/how_to/qa_sources.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
"## Setup\n",
"### Dependencies\n",
"\n",
"We’ll use an OpenAI chat model and embeddings and a Memory vector store in this walkthrough, but everything shown here works with any [ChatModel](/docs/concepts/#chat-models) or [LLM](/docs/modules/model_io/llms), [Embeddings](/docs/modules/data_connection/text_embedding/), and [VectorStore](/docs/modules/data_connection/vectorstores/) or [Retriever](/docs/modules/data_connection/retrievers/).\n",
"We’ll use an OpenAI chat model and embeddings and a Memory vector store in this walkthrough, but everything shown here works with any [ChatModel](/docs/concepts/#chat-models) or [LLM](/docs/concepts#llms), [Embeddings](/docs/concepts#embedding-models), and [VectorStore](/docs/concepts#vectorstores) or [Retriever](/docs/concepts#retrievers).\n",
"\n",
"We’ll use the following packages:\n",
"\n",
Expand Down Expand Up @@ -79,7 +79,7 @@
},
{
"cell_type": "code",
"execution_count": 10,
"execution_count": 3,
"metadata": {},
"outputs": [],
"source": [
Expand Down Expand Up @@ -109,6 +109,33 @@
"]);"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's see what this prompt actually looks like:"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"You are an assistant for question-answering tasks. Use the following pieces of retrieved context to answer the question. If you don't know the answer, just say that you don't know. Use three sentences maximum and keep the answer concise.\n",
"Question: {question} \n",
"Context: {context} \n",
"Answer:\n"
]
}
],
"source": [
"console.log(prompt.promptMessages.map((msg) => msg.prompt.template).join(\"\\n\"));"
]
},
{
"cell_type": "code",
"execution_count": 11,
Expand Down
31 changes: 29 additions & 2 deletions docs/core_docs/docs/how_to/qa_streaming.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
"## Setup\n",
"### Dependencies\n",
"\n",
"We’ll use an OpenAI chat model and embeddings and a Memory vector store in this walkthrough, but everything shown here works with any [ChatModel](/docs/concepts/#chat-models) or [LLM](/docs/modules/model_io/llms), [Embeddings](/docs/modules/data_connection/text_embedding/), and [VectorStore](/docs/modules/data_connection/vectorstores/) or [Retriever](/docs/modules/data_connection/retrievers/).\n",
"We’ll use an OpenAI chat model and embeddings and a Memory vector store in this walkthrough, but everything shown here works with any [ChatModel](/docs/concepts/#chat-models) or [LLM](/docs/concepts#llms), [Embeddings](/docs/concepts#embedding-models), and [VectorStore](/docs/concepts#vectorstores) or [Retriever](/docs/concepts#retrievers).\n",
"\n",
"We’ll use the following packages:\n",
"\n",
Expand Down Expand Up @@ -124,7 +124,7 @@
" }\n",
" }\n",
" ],\n",
" answer: \u001b[32m\"Task decomposition is a technique used to break down complex tasks into smaller and simpler steps. I\"\u001b[39m... 256 more characters\n",
" answer: \u001b[32m\"Task decomposition is a technique used to break down complex tasks into smaller and simpler steps fo\"\u001b[39m... 230 more characters\n",
"}"
]
},
Expand Down Expand Up @@ -162,6 +162,33 @@
"await ragChainWithSource.invoke(\"What is Task Decomposition\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's see what this prompt actually looks like:"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"You are an assistant for question-answering tasks. Use the following pieces of retrieved context to answer the question. If you don't know the answer, just say that you don't know. Use three sentences maximum and keep the answer concise.\n",
"Question: {question} \n",
"Context: {context} \n",
"Answer:\n"
]
}
],
"source": [
"console.log(prompt.promptMessages.map((msg) => msg.prompt.template).join(\"\\n\"));"
]
},
{
"cell_type": "markdown",
"metadata": {},
Expand Down
2 changes: 1 addition & 1 deletion docs/core_docs/docs/how_to/vectorstore_retriever.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ the specific retriever object you are calling.

Of course, we also help construct what we think useful Retrievers are. The main type of Retriever in LangChain is a vector store retriever. We will focus on that here.

**Note:** Before reading, it's important to understand [what a vector store is](/docs/modules/data_connection/vectorstores).
**Note:** Before reading, it's important to understand [what a vector store is](/docs/concepts#vectorstores).

This example showcases question answering over documents.
We have chosen this as the example for getting started because it nicely combines a lot of different elements (Text splitters, embeddings, vectorstores) and then also shows how to use them in a chain.
Expand Down
2 changes: 1 addition & 1 deletion docs/core_docs/docs/how_to/vectorstores.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ for you.

## Get started

This walkthrough showcases basic functionality related to VectorStores. A key part of working with vector stores is creating the vector to put in them, which is usually created via embeddings. Therefore, it is recommended that you familiarize yourself with the [text embedding model](/docs/modules/data_connection/text_embedding/) interfaces before diving into this.
This walkthrough showcases basic functionality related to VectorStores. A key part of working with vector stores is creating the vector to put in them, which is usually created via embeddings. Therefore, it is recommended that you familiarize yourself with the [text embedding model](/docs/concepts#embedding-models) interfaces before diving into this.

This walkthrough uses a basic, unoptimized implementation called MemoryVectorStore that stores embeddings in-memory and does an exact, linear search for the most similar embeddings.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ hide_table_of_contents: true

# Vector Store

Once you've created a [Vector Store](/docs/modules/data_connection/vectorstores), the way to use it as a Retriever is very simple:
Once you've created a [Vector Store](/docs/concepts#vectorstores), the way to use it as a Retriever is very simple:

```typescript
vectorStore = ...
Expand Down
27 changes: 27 additions & 0 deletions docs/core_docs/docs/tutorials/local_rag.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -320,6 +320,33 @@
"});"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's see what this prompt actually looks like:"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"You are an assistant for question-answering tasks. Use the following pieces of retrieved context to answer the question. If you don't know the answer, just say that you don't know. Use three sentences maximum and keep the answer concise.\n",
"Question: {question} \n",
"Context: {context} \n",
"Answer:\n"
]
}
],
"source": [
"console.log(ragPrompt.promptMessages.map((msg) => msg.prompt.template).join(\"\\n\"));"
]
},
{
"cell_type": "code",
"execution_count": 8,
Expand Down
29 changes: 28 additions & 1 deletion docs/core_docs/docs/tutorials/qa_chat_history.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@
"## Setup\n",
"### Dependencies\n",
"\n",
"We’ll use an OpenAI chat model and embeddings and a Memory vector store in this walkthrough, but everything shown here works with any [ChatModel](/docs/concepts/#chat-models) or [LLM](/docs/modules/model_io/llms), [Embeddings](/docs/modules/data_connection/text_embedding/), and [VectorStore](/docs/modules/data_connection/vectorstores/) or [Retriever](/docs/modules/data_connection/retrievers/).\n",
"We’ll use an OpenAI chat model and embeddings and a Memory vector store in this walkthrough, but everything shown here works with any [ChatModel](/docs/concepts/#chat-models) or [LLM](/docs/concepts#llms), [Embeddings](/docs/concepts#embedding-models), and [VectorStore](/docs/concepts#vectorstores) or [Retriever](/docs/concepts#retrievers).\n",
"\n",
"We’ll use the following packages:\n",
"\n",
Expand Down Expand Up @@ -110,6 +110,33 @@
"});"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's see what this prompt actually looks like:"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"You are an assistant for question-answering tasks. Use the following pieces of retrieved context to answer the question. If you don't know the answer, just say that you don't know. Use three sentences maximum and keep the answer concise.\n",
"Question: {question} \n",
"Context: {context} \n",
"Answer:\n"
]
}
],
"source": [
"console.log(prompt.promptMessages.map((msg) => msg.prompt.template).join(\"\\n\"));"
]
},
{
"cell_type": "code",
"execution_count": 7,
Expand Down
Loading
Loading