Skip to content

update OpenAI function_call to tool_call (#8015) #8016

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

sajozsattila
Copy link

Updated the OpenAI evaluation model. For details see #8015 bug

@sajozsattila sajozsattila requested a review from a team as a code owner June 10, 2025 15:18
@github-project-automation github-project-automation bot moved this to 📘 Todo in phoenix Jun 10, 2025
@dosubot dosubot bot added the size:S This PR changes 10-29 lines, ignoring generated files. label Jun 10, 2025
Copy link
Contributor

CLA Assistant Lite bot:
Thank you for your submission, we really appreciate it. Like many open-source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution. You can sign the CLA by just posting a Pull Request Comment same as the below format.


I have read the CLA Document and I hereby sign the CLA


Sajo Zsolt Attila seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You can retrigger this bot by commenting recheck in this Pull Request

Copy link
Contributor

@anticorrelator anticorrelator left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @sajozsattila thanks so much for the contribution!

In order to fully support this we also need to wire up tool_choice into llm_classify and run_evals. Specifically we need to modify the openai_function_call_kwargs function to pass in the new tool_choice kwarg into the model. Furthermore we should consider deprecating the use_function_calling_if_available kwarg in preference for the use_tool_calling_if_available keyword argument.

Please include an example that shows the modified behavior still works after the changes either as a small script in the PR description that can be replicated or a unit test as well

@github-project-automation github-project-automation bot moved this from 📘 Todo to 🔍. Needs Review in phoenix Jun 10, 2025
@dosubot dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. and removed size:S This PR changes 10-29 lines, ignoring generated files. labels Jun 10, 2025
@@ -101,7 +101,7 @@ def llm_classify(
data_processor: Optional[Callable[[PROCESSOR_TYPE], PROCESSOR_TYPE]] = None,
system_instruction: Optional[str] = None,
verbose: bool = False,
use_function_calling_if_available: bool = True,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should leave the old kwarg around as well and raise a warning if users try to use it. I don't think this update is worth breaking backwards compatibility for

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's no substantive advantage in changing the name of parameters since "function calling" remains a valid terminology: I would recommend that we keep the old parameter instead.

@@ -368,7 +368,7 @@ def run_evals(
dataframe: DataFrame,
evaluators: List[LLMEvaluator],
provide_explanation: bool = False,
use_function_calling_if_available: bool = True,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same as above

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

@RogerHYang
Copy link
Contributor

RogerHYang commented Jun 10, 2025

It's not necessary to rename function_call to tool_call everywhere, because "function calling" remains a valid terminology. The real change that's needed to revise the kwargs here, i.e. "functions" should now be "tools" and "function_call" should now be "tool_choice" with the value of {"type": "function", "function": {"name": openai_function["name"]}}.

@sajozsattila
Copy link
Author

I made some changes, please check it. There is also a script to check. You need to set the OpenAI or Openrouter key to test.

phoenix_opeanai_tool_test.tgz

@RogerHYang
Copy link
Contributor

replace this with the following

    return {
        "tools": [openai_function],
        "tool_choice": {"type": "function", "function": {"name": openai_function["name"]}},
    }

@RogerHYang
Copy link
Contributor

Sajo Zsolt Attila seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.

@sajozsattila there seems to be an issue with mismatching emails associated with your commit. See message above. Are you able to fix?

@sajozsattila
Copy link
Author

I am not moving forward with this pull request at the moment. I initially thought it was just a quick fix, but it seems I need to spend more time with the code than I currently have.

@sajozsattila sajozsattila marked this pull request as draft June 11, 2025 07:42
@RogerHYang
Copy link
Contributor

RogerHYang commented Jun 11, 2025

@sajozsattila No worries, we can take over this PR and get it merged. Thanks for bringing this to our attention.

On the other hand, we would love to add you as a contributor. For that to work, would you be able to add your email to your Github account, so that you can accept the CLA as noted above?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
size:L This PR changes 100-499 lines, ignoring generated files.
Projects
Status: 🔍. Needs Review
Development

Successfully merging this pull request may close these issues.

3 participants