Skip to content

Qdrant Wrapper issue: _document_from_score_point exposes incorrect key for content #1087

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
rishabh-ti opened this issue Feb 16, 2023 · 5 comments · Fixed by #1088
Closed

Comments

@rishabh-ti
Copy link
Contributor

Screenshot 2023-02-16 at 6 47 59 PM

pydantic.error_wrappers.ValidationError: 1 validation error for Document
page_content
  none is not an allowed value (type=type_error.none.not_allowed)
hwchase17 pushed a commit that referenced this issue Feb 16, 2023
@kacperlukawski
Copy link
Contributor

kacperlukawski commented Feb 16, 2023

@rishabh-ti Could you please share some more context? How do you create a Qdrant collection? Are there any chances you provide None as one of the texts?

hwchase17 pushed a commit that referenced this issue Feb 16, 2023
The #1088 introduced a bug in Qdrant integration. That PR reverts those
changes and provides class attributes to ensure consistent payload keys.
In addition to that, an exception will be thrown if any of texts is None
(that could have been an issue reported in #1087)
@rishabh-ti
Copy link
Contributor Author

rishabh-ti commented Feb 17, 2023

@kacperlukawski

client = QdrantClient(host="localhost", port=6333)
embeddings = OpenAIEmbeddings()
qdrant = Qdrant(client=client,collection_name=collection_name,embedding_function=embeddings.embed_query)
llm = OpenAI(model_name="text-davinci-003", temperature=0.0, max_tokens=1500)
qa_chain = ChatVectorDBChain.from_llm(
    llm=llm,
    vectorstore=qdrant,
    condense_question_prompt=question_prompt,
    qa_prompt=answer_prompt,
    chain_type="stuff"
)

So my collection & records already exist in Qdrant. I was passing it in ChatVectorDBChain which was giving me issues when I ran:
result = qa_chain({"chat_history": chat_history,"question": query})

@kacperlukawski
Copy link
Contributor

kacperlukawski commented Feb 17, 2023

@rishabh-ti Great to see the whole context. Thanks a lot! I assume you uploaded the records on your own and just used the "content" payload key to store the texts. If so, we can consider introducing two additional parameters into the __init__ of Qdrant:

  1. Document content payload key
  2. Metadata payload key

The default values would be set to "page_content" and "metadata" respectively, but in your case, you'll be able to change them so they work as intended. Would that be ok?

@rishabh-ti
Copy link
Contributor Author

@rishabh-ti Great to see the whole context. Thanks a lot! I assume you uploaded the records on your own and just used the "content" payload key to store the texts. If so, we can consider introducing two additional parameters into the __init__ of Qdrant:

  1. Document content payload key
  2. Metadata payload key

The default values would be set to "page_content" and "metadata" respectively, but in your case, you'll be able to change them so they work as intended. Would that be ok?

Yes that would work!

dongreenberg pushed a commit to dongreenberg/langchain that referenced this issue Feb 17, 2023
hwchase17 pushed a commit that referenced this issue Mar 2, 2023
This PR:
- Increases `qdrant-client` version to 1.0.4
- Introduces custom content and metadata keys (as requested in #1087)
- Moves all the `QdrantClient` parameters into the method parameters to
simplify code completion
zachschillaci27 pushed a commit to zachschillaci27/langchain that referenced this issue Mar 8, 2023
zachschillaci27 pushed a commit to zachschillaci27/langchain that referenced this issue Mar 8, 2023
…in-ai#1093)

The langchain-ai#1088 introduced a bug in Qdrant integration. That PR reverts those
changes and provides class attributes to ensure consistent payload keys.
In addition to that, an exception will be thrown if any of texts is None
(that could have been an issue reported in langchain-ai#1087)
zachschillaci27 pushed a commit to zachschillaci27/langchain that referenced this issue Mar 8, 2023
This PR:
- Increases `qdrant-client` version to 1.0.4
- Introduces custom content and metadata keys (as requested in langchain-ai#1087)
- Moves all the `QdrantClient` parameters into the method parameters to
simplify code completion
@Thiru-GVT
Copy link

Hello, I have this exact same issue. Could I ask you to expand more on this metadata?

I have my qdrant server with data inside, but when i try to use it as a retriever, i keep getting this validation error

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants