Auto select functions for Chat Completion

[sashirestela]

This is an extra feature whose benefit was identified in:

openai/openai-openapi#259 (comment)

[sashirestela]

This could be a helper class that would store the list of all the tools and would expose a method to feed the ChatRequest.tools field, with the following signature:

List<Tool> functions(Object toolChoice) {
    ...
}

The parameter would have the same value being used for the ChatRequest.toolChoice field: a ToolChoiceOption enum or a ToolChoice object.

The logic of the new method functions(toolChoice) would be:

• If the toolChoice is ToolChoiceOption.NONE, then it returns an empty list of tools.
• If it is ToolChoiceOption.AUTO or ToolChoiceOption.REQUIRED, then it returns the entire list of tools.
• If it is ToolChoice with a function name, then it returns a list with that function only.

@the-gigi, Am I missing something?

NOTE: By the way, it is supposed that as a user of this library, you will be responsible to analyze the context in every chat cycle and determine the correct value for the toolChoice parameter, right?

[the-gigi]

The logic of the new method functions(toolChoice) would be:

• If the toolChoice is ToolChoiceOption.NONE, then it returns an empty list of tools.
• If it is ToolChoiceOption.AUTO or ToolChoiceOption.REQUIRED, then it returns the entire list of tools.
• If it is ToolChoice with a function name, then it returns a list with that function only.

@the-gigi, Am I missing something?

This covers all the cases. I assume the helper class is going to be used internally. right? The user doesn't have to call it before populating a ChatRequest.

[sashirestela]

I think that the FunctionExecutor class is the natural option to provide this functionality, because it currently stores the function definitions. The high level pseudo code would be something like this:

// create a list of all your functions
List<FunctionDef> functionList = new ArrayList<>()
functionList.add(functionDefinition1)
functionList.add(functionDefinition2)
...
functionList.add(functionDefinitionN)

// create the function executor and populate it
var functionExecutor = new FunctionExecutor(functionList)

// create the list of messages
List<ChatMessage> messages = new ArrayList<>()

while (continueConversation) {

    myToolChoice = ... // calculate it based off the current conversation context
    
    // create the chat request as usual passing on parameters, including tools and toolChoice
    chatRequest = ChatRequest.builder()
        .tools(functionExecutor.getToolFunctions(myToolChoice))  // getToolFunctions already exists ...
        .toolChoice(myToolChoice)                                // ... and it will be overloaded
        .messages(messages)
        .otherParameters(...)
        .build()

    // call chat completions service
    var chatStream = openAI.chatCompletions().createStream(chatRequest).join()
    
    // process response and add chat messages
    ...
                    
}

[sashirestela]

@the-gigi any concern with the proposal?

[the-gigi]

@sashirestela No concern. It keeps the API the same, so users are not impacted and the FunctionExecutor encapsulates the logic of pruning the tools according to the user's tool choice. I think it's a clean design.