-
Notifications
You must be signed in to change notification settings - Fork 490
update OpenAI function_call to tool_call (#8015) #8016
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
CLA Assistant Lite bot: I have read the CLA Document and I hereby sign the CLA Sajo Zsolt Attila seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi @sajozsattila thanks so much for the contribution!
In order to fully support this we also need to wire up tool_choice
into llm_classify
and run_evals
. Specifically we need to modify the openai_function_call_kwargs
function to pass in the new tool_choice
kwarg into the model. Furthermore we should consider deprecating the use_function_calling_if_available
kwarg in preference for the use_tool_calling_if_available
keyword argument.
Please include an example that shows the modified behavior still works after the changes either as a small script in the PR description that can be replicated or a unit test as well
@@ -101,7 +101,7 @@ def llm_classify( | |||
data_processor: Optional[Callable[[PROCESSOR_TYPE], PROCESSOR_TYPE]] = None, | |||
system_instruction: Optional[str] = None, | |||
verbose: bool = False, | |||
use_function_calling_if_available: bool = True, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we should leave the old kwarg around as well and raise a warning if users try to use it. I don't think this update is worth breaking backwards compatibility for
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fixed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There's no substantive advantage in changing the name of parameters since "function calling" remains a valid terminology: I would recommend that we keep the old parameter instead.
@@ -368,7 +368,7 @@ def run_evals( | |||
dataframe: DataFrame, | |||
evaluators: List[LLMEvaluator], | |||
provide_explanation: bool = False, | |||
use_function_calling_if_available: bool = True, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
same as above
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fixed
It's not necessary to rename |
I made some changes, please check it. There is also a script to check. You need to set the OpenAI or Openrouter key to test. |
replace this with the following return {
"tools": [openai_function],
"tool_choice": {"type": "function", "function": {"name": openai_function["name"]}},
} |
@sajozsattila there seems to be an issue with mismatching emails associated with your commit. See message above. Are you able to fix? |
I am not moving forward with this pull request at the moment. I initially thought it was just a quick fix, but it seems I need to spend more time with the code than I currently have. |
@sajozsattila No worries, we can take over this PR and get it merged. Thanks for bringing this to our attention. On the other hand, we would love to add you as a contributor. For that to work, would you be able to add your email to your Github account, so that you can accept the CLA as noted above? |
Updated the OpenAI evaluation model. For details see #8015 bug