Pass tenant parameter when creating `WeaviateVectorStore` directly

[joshanderson-sas]

Is your feature request related to a problem? Please describe.
Currently there is no way to pass a tenant parameter when creating a WeaviateVectorStore object directly. The only solution currently appears to be using the from_documents class method, like this.

db_with_mt = WeaviateVectorStore.from_documents(
    docs, embeddings, client=weaviate_client, tenant="Foo"
)

Describe the solution you'd like
Would like to be able to pass a tenant when instantiating a WeaviateVectorStore, like this.

db = WeaviateVectorStore(embedding=embeddings, client=client, index_name="Collection", text_key="text", tenant = "Foo")

[hsm207]

@joshanderson-sas 2 questions to clarify the feature request:

1. List of tenants
   You actually want to be able to pass a list of tenants, correct? So, your example should be:

db = WeaviateVectorStore(embedding=embeddings, client=client, index_name="Collection", text_key="text", tenant = ["Foo"])

2. What happens when a user passes a different tenant when doing CRUD ops?

For example:

db = WeaviateVectorStore(embedding=embeddings, client=client, index_name="Collection", text_key="text", tenant = ["Foo"])

db.similarity_search(query, tenant="Bar")

[joshanderson-sas]

Hi @hsm207, thanks for the quick response. Just to make sure I understand tenants properly, am I understanding correctly that if I have two tenants, A and B, and I call add_texts with tenant A on an index with some documents data from source C, that if tenant B performed a similarity search on, they would not get results from the source C documents?

For #1, yes, a list of tenants makes sense. I'm thinking that these tenants could be the default tenants used for actions like, add_texts or similarity_search. So if you provide the tenant param, use that tenant. Otherwise use the default list of tenants.

For #2, could this be handled in the same way as if you called add_texts with tenant A and similarity_search with tenant B? Would there be some "Tenant does not exist" error in this case?

The use case scenario I'm imagining is like this:

Creating the doc index

vector_store = WeaviateVectorStore(
    embedding=embeddings, client=client, index_name="Collection", text_key="text", tenant = ["A", "B"]
)

# Add documents to db with langchain record manager
# Documents indexed for tenants "A" and "B" 
rm.indexing(docs, collection, db)

Querying the index

vector_store = WeaviateVectorStore(
    embedding=embeddings, client=client, index_name="Collection", text_key="text", tenant = ["A"]
)

# Tenant A was used previously, so this should be fine
# Tenant B would also work
# Tenant C could throw runtime error?
retriever = vector_store.as_retriever()
chain = create_retrieval_chain(retriever, qa_chain)

# Handle RAG query

[hsm207]

if I have two tenants, A and B, and I call add_texts with tenant A on an index with some documents data from source C, that if tenant B performed a similarity search on, they would not get results from the source C documents?

If a search is performed on tenant B, then only the documents belonging to tenant B are considered in the search.

So if you provide the tenant param, use that tenant. Otherwise use the default list of tenants.

This can be a security risk. Imagine if each tenant is a distinct customer and the developer forgot to provide the tenant param. Then, the any query will include results across all customers. Or imagine a delete operation ...

For #2, could this be handled in the same way as if you called add_texts with tenant A and similarity_search with tenant B? Would there be some "Tenant does not exist" error in this case?

It depends if you have auto tenant creation turned on. It is turned off by default so if you try to do CRUD into a tenant that does not exist, it will throw an error.

vector_store = WeaviateVectorStore(
    embedding=embeddings, client=client, index_name="Collection", text_key="text", tenant = ["A", "B"]
)

# Add documents to db with langchain record manager
# Documents indexed for tenants "A" and "B" 
rm.indexing(docs, collection, db)

I don't think rm.indexing(docs, collection, db) is ideal. What if the user wants to index docs into tenant A only?

vector_store = WeaviateVectorStore(
    embedding=embeddings, client=client, index_name="Collection", text_key="text", tenant = ["A"]
)

# Tenant A was used previously, so this should be fine
# Tenant B would also work
# Tenant C could throw runtime error?
retriever = vector_store.as_retriever()
chain = create_retrieval_chain(retriever, qa_chain)

# Handle RAG query

how should the chain's query method look? When MT is turned on, you must specify the tenant you want to query

[joshanderson-sas]

Thanks for the info. Closing this for now. I need to do some more experiments with the multi-tenancy options and understand better how it works.

[hsm207]

@joshanderson-sas reopening this to discuss your requirements for MT

my understanding is that you'd like WeaviateVectorStore to take a tenants param that is a list of strings (tenant names) that weaviate will use to create the tenants.

would there be any changes required to the search and delete interface too?