Expanded Support for Vectara's Search API

[Kae7in]

Add support for Context Config, Generative Summarization, and more

This PR gives users of LangChainJS's Vectara integration more fine-grained control over Vectara's Search API.

New similaritySearchWithSummary() method allows for use of Vectara's Generative Summarization feature.

New optional fields on similaritySearch() methods:

• The contextConfig exposes additional query context properties that the system may use to refine the results.
• The start field controls the starting position within the list of results.
• The rerankingConfig enables reranking of results, to further increase relevance in certain scenarios.
• The summary field allows you to use Vectara's "Gounded Generation", a groundbreaking way of producing generative summaries on top of your own data.

Other additions:

• An optional similaritySearchWithSummary(...) method has been added to the base VectorStore class, which also paves the way for other Vector databases that might implement similar summarization techniques in the near future.
• Tests added for all of the above.

---

About me:

• Twitter handle: @KaelinHooper
• Website: kaelin.io

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Updated (UTC)
langchainjs-docs	✅ Ready (Inspect)	Visit Preview	Aug 20, 2023 4:00am

[dosubot]

This PR adds a new method similaritySearchWithSummary that makes an external HTTP request using fetch to the Vectara API endpoint. This comment is flagging the change for maintainers to review the addition of the new request.

[jacoblee93]

This is great - do you think this could make sense as a new searchType like MMR on MongoDB?

https://github.com/hwchase17/langchainjs/pull/2170/files

That way, you could initialize a retriever in this mode and it would automatically call similaritySearchWithSummary

If the summary functionality is also possible with specific args to similaritySearch or you don't think it's worth it, let's not add this method to the base class.

[Kae7in]

This is great - do you think this could make sense as a new searchType like MMR on MongoDB?

https://github.com/hwchase17/langchainjs/pull/2170/files

That way, you could initialize a retriever in this mode and it would automatically call similaritySearchWithSummary

If the summary functionality is also possible with specific args to similaritySearch or you don't think it's worth it, let's not add this method to the base class.

Thanks for suggesting this! I'll take a look at this approach soon. I definitely didn't feel great about adding an entire method that will throw an error by default for most VectorStore integrations. I'll see about making this better. Might ping you with some questions.

[danielchalef]

+1 for a new search type. We'll be adding this to Zep soon.

[ofermend]

+1 from me as well Happy to help with this PR too if needed. Pls lmk.

…

On Tue, Aug 15, 2023 at 4:27 PM Daniel Chalef ***@***.***> wrote: +1 for a new search type. We'll be adding this to Zep soon. — Reply to this email directly, view it on GitHub <#2206 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAN5GO7BLRJZE6KLOOTFWQTXVQAV3ANCNFSM6AAAAAA3IIYHCI> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[jacoblee93]

Hey @Kae7in is this ready for review?

[ofermend]

Hey @Kae7in , @jacoblee93 - any updates on this?
Pls let me know if we can help

[ofermend]

Update?