Skip to content

feature(community): Enable using chat and llms without providing project/space/development id (lightweight engine IBM) #7781

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 13 commits into from
Mar 11, 2025
Merged

feature(community): Enable using chat and llms without providing project/space/development id (lightweight engine IBM) #7781

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
818c888
feature: Add chat deployment to chat class (#41)
FilipZmijewski Jan 30, 2025
f818281
fix: Add fixes regarding PR comments, change models in tests (#42)
FilipZmijewski Feb 4, 2025
0dc5cac
fix: Remove optional checker
FilipZmijewski Feb 4, 2025
450f6cd
fix: Fix deployment chat interface (#44)
FilipZmijewski Feb 7, 2025
dd83aa9
Merge remote-tracking branch 'upstream/main'
FilipZmijewski Feb 7, 2025
b1ad066
Fix test
FilipZmijewski Feb 7, 2025
bb3c2cf
Rework tests without langgraph
FilipZmijewski Feb 10, 2025
ebe39f9
fix: Rework tests without langgraph (#45)
FilipZmijewski Feb 10, 2025
9a06b28
Merge branch 'main' of https://github.com/FilipZmijewski/langchainjs
FilipZmijewski Feb 28, 2025
b9f65ac
Merge branch 'main' of https://github.com/langchain-ai/langchainjs
FilipZmijewski Feb 28, 2025
fa73d2f
feat: Add support for lightweight engine
FilipZmijewski Feb 28, 2025
e953757
Change id in notebooks
FilipZmijewski Feb 28, 2025
dfae651
Enable lightweight engine in IBM implementation
FilipZmijewski Mar 7, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions docs/core_docs/docs/integrations/chat/ibm.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -171,7 +171,8 @@
" version: \"YYYY-MM-DD\",\n",
" serviceUrl: process.env.API_URL,\n",
" projectId: \"<PROJECT_ID>\",\n",
" spaceId: \"<SPACE_ID>\",\n",
" // spaceId: \"<SPACE_ID>\",\n",
" // idOrName: \"<DEPLOYMENT_ID>\",\n",
" model: \"<MODEL_ID>\",\n",
" ...props\n",
"});"
Expand All @@ -184,7 +185,7 @@
"source": [
"Note:\n",
"\n",
"- You must provide `spaceId` or `projectId` in order to proceed.\n",
"- You must provide `spaceId`, `projectId` or `idOrName`(deployment id) unless you use lighweight engine which works without specifying either (refer to [watsonx.ai docs](https://www.ibm.com/docs/en/cloud-paks/cp-data/5.0.x?topic=install-choosing-installation-mode))\n",
"- Depending on the region of your provisioned service instance, use correct serviceUrl."
]
},
Expand Down
6 changes: 3 additions & 3 deletions docs/core_docs/docs/integrations/llms/ibm.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -171,8 +171,8 @@
" version: \"YYYY-MM-DD\",\n",
" serviceUrl: process.env.API_URL,\n",
" projectId: \"<PROJECT_ID>\",\n",
" spaceId: \"<SPACE_ID>\",\n",
" idOrName: \"<DEPLOYMENT_ID>\",\n",
" // spaceId: \"<SPACE_ID>\",\n",
" // idOrName: \"<DEPLOYMENT_ID>\",\n",
" model: \"<MODEL_ID>\",\n",
" ...props,\n",
"});"
Expand All @@ -185,7 +185,7 @@
"source": [
"Note:\n",
"\n",
"- You must provide `spaceId`, `projectId` or `idOrName`(deployment id) in order to proceed.\n",
"- You must provide `spaceId`, `projectId` or `idOrName`(deployment id) unless you use lighweight engine which works without specifying either (refer to [watsonx.ai docs](https://www.ibm.com/docs/en/cloud-paks/cp-data/5.0.x?topic=install-choosing-installation-mode))\n",
"- Depending on the region of your provisioned service instance, use correct serviceUrl.\n",
"- You need to specify the model you want to use for inferencing through model_id."
]
Expand Down
14 changes: 7 additions & 7 deletions libs/langchain-community/src/chat_models/ibm.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -465,11 +465,6 @@ export class ChatWatsonx<
)
throw new Error("Maximum 1 id type can be specified per instance");

if (!("projectId" in fields || "spaceId" in fields || "idOrName" in fields))
throw new Error(
"No id specified! At least id of 1 type has to be specified"
);
Comment on lines -468 to -471
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we keep this check and only bypass it when the lightweight engine is in use?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So that would enforce us to use a flag ex. lightweight: true. Unless user uses a lightweight engine the API is going to throw an error "No spaceId nor projectId is present" so this is taken care of there.
Also, we want to match with our Python Langchain SDK implementation. Assuming whenever id is not passed user chooses lightweight engine. If id of any type is mistakenly not passed the API will throw an error letting user know.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@FilipZmijewski just to clarify, are you saying that the API does check this and therefore the check in our code is unnecessary? The benefit of having the check in our code is that it causes us to throw a nice/readable error. Is there no other way that we can infer that the user is using the lightweight engine?

By the way, feel free to reply in your native language if you're more comfortable that way. LLMs are very good at translating these days 😄

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@benjamincburns We would like to match the solution with what we have in SDK (both python and node.js) and also langchain IBM python meaning "no id provided -> lightweight engine in use". In case of lightweight engine unavailable the API will throw an error, quite clear to understand, Missing either space_id or project_id or wml_instance_crn. One thing that could be added is disabling error check depending on serviceUrl provided but this is not ideal since not all IBM watsonx.ai software instalations will allow user to use lightweight engine.

Haha, thanks. I enjoy speaking/writing in English so that works!


if ("model" in fields) {
this.projectId = fields?.projectId;
this.spaceId = fields?.spaceId;
Expand Down Expand Up @@ -564,13 +559,18 @@ export class ChatWatsonx<
scopeId():
| { idOrName: string }
| { projectId: string; modelId: string }
| { spaceId: string; modelId: string } {
| { spaceId: string; modelId: string }
| { modelId: string } {
if (this.projectId && this.model)
return { projectId: this.projectId, modelId: this.model };
else if (this.spaceId && this.model)
return { spaceId: this.spaceId, modelId: this.model };
else if (this.idOrName) return { idOrName: this.idOrName };
else throw new Error("No scope id provided");
else if (this.model)
return {
modelId: this.model,
};
else throw new Error("No id or model provided!");
}

async completionWithRetry<T>(
Expand Down
16 changes: 7 additions & 9 deletions libs/langchain-community/src/chat_models/tests/ibm.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -141,24 +141,22 @@ describe("LLM unit tests", () => {

testProperties(instance, testProps);
});
});

describe("Negative tests", () => {
test("Missing id", async () => {
const testProps: ChatWatsonxInput = {
model: "mistralai/mistral-large",
version: "2024-05-31",
serviceUrl: process.env.WATSONX_AI_SERVICE_URL as string,
};
expect(
() =>
new ChatWatsonx({
...testProps,
...fakeAuthProp,
})
).toThrowError();
const intance = new ChatWatsonx({
...testProps,
...fakeAuthProp,
});
expect(intance).toBeDefined();
});
});

describe("Negative tests", () => {
test("Missing other props", async () => {
// @ts-expect-error Intentionally passing not enough parameters
const testPropsProjectId: ChatWatsonxInput = {
Expand Down
7 changes: 1 addition & 6 deletions libs/langchain-community/src/llms/ibm.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -184,11 +184,6 @@ export class WatsonxLLM<
)
throw new Error("Maximum 1 id type can be specified per instance");

if (!("projectId" in fields || "spaceId" in fields || "idOrName" in fields))
throw new Error(
"No id specified! At least id of 1 type has to be specified"
);
Comment on lines -187 to -190
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same question here


this.serviceUrl = fields?.serviceUrl;
const {
watsonxAIApikey,
Expand Down Expand Up @@ -281,7 +276,7 @@ export class WatsonxLLM<
return { spaceId: this.spaceId, modelId: this.model };
else if (this.idOrName)
return { idOrName: this.idOrName, modelId: this.model };
else return { spaceId: this.spaceId, modelId: this.model };
else return { modelId: this.model };
}

async listModels() {
Expand Down
28 changes: 7 additions & 21 deletions libs/langchain-community/src/llms/tests/ibm.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -125,24 +125,21 @@ describe("LLM unit tests", () => {

testProperties(instance, testProps);
});
});

describe("Negative tests", () => {
test("Missing id", async () => {
const testProps: WatsonxInputLLM = {
model: "ibm/granite-13b-chat-v2",
version: "2024-05-31",
serviceUrl: process.env.WATSONX_AI_SERVICE_URL as string,
};
expect(
() =>
new WatsonxLLM({
...testProps,
...fakeAuthProp,
})
).toThrowError();
const instance = new WatsonxLLM({
...testProps,
...fakeAuthProp,
});
expect(instance).toBeDefined();
});
});

describe("Negative tests", () => {
test("Missing other props", async () => {
// @ts-expect-error Intentionally passing not enough parameters
const testPropsProjectId: WatsonxInputLLM = {
Expand All @@ -156,17 +153,6 @@ describe("LLM unit tests", () => {
...fakeAuthProp,
})
).toThrowError();
// @ts-expect-error Intentionally passing not enough parameters
const testPropsServiceUrl: WatsonxInputLLM = {
serviceUrl: process.env.WATSONX_AI_SERVICE_URL as string,
};
expect(
() =>
new WatsonxLLM({
...testPropsServiceUrl,
...fakeAuthProp,
})
).toThrowError();
Comment on lines -159 to -169
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assuming it's possible to check for the lightweight engine, you could keep this logic here, but pass whatever is necessary to detect that use

const testPropsVersion = {
version: "2024-05-31",
};
Expand Down
Loading