Handle undocumented tools such as browser
#283
Assignees
Labels
enhancement
New feature or request
Comments
|
Hi @amavashev. Could you share your request to replicate the issue? UPDATE: Looking at the stack trace error, OpenAI could be breaking the API because is sending back an unknown, undocumented value ( https://platform.openai.com/docs/api-reference/run-steps/step-object#run-steps/step-object-step_details, tool_calls |
|
We use our own custom action: web_search, which basically does web search and returns results via text to action. But dont have exact request we sent, we are sending so many. Also it does not happen all the time, just somtimes for the same web search query. So not sure how to reproduce, happens sporadically. |
|
@amavashev Without a request, I cannot replicate your scenario. Said that and based on the error message, what I see is:
So, it seems the error is not related to simple-openai |
|
The steps are correct. seems like openAI send back parms which simpl-openai can't handle. Maybe there is more flexible way to handle this in the API. It is definitely the case that openAI returns and undocumented parms and if so, shouldn't the API handle that and not fail? Also anyway to log all requests and responses sent by simple-openai? |
|
An API is a contract that both parties expect to honor, but if you don't honor it, the API loses its meaning. simple-openai has been developed with a degree of resilience to allow for unknown attributes, but I can't develop the entire library thinking that the API won't be minimally honored, it just doesn't make sense. You should report this case to the OpenAI developer forum, but I recommend that you attach both the request and the response. Regarding logging, it depends on the logging library that you are using. If you want to see an example you could take a look at simple-openai-playground where I'm using slf4j-simple. For example if you run the following command: mvn exec:java -Dexec.mainClass=io.github.sashirestela.openai.playground.DemoVision -Dorg.slf4j.simpleLogger.properties=src/main/resources/simplelogger.propertiesYou will get the following log file: example.log
|
|
@amavashev |
sashirestela
changed the title
browser
Merged
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
io.github.sashirestela.cleverclient.support.CleverClientException: Cannot convert the Json {"id":"step_dVc3iOEPBmhFEP9DV3cosMMj","object":"thread.run.step.delta","delta":{"step_details":{"type":"tool_calls","tool_calls":[{"index":0,"id":"call_rwX78tZl6MzaPfNMYu5fFsQD","type":"browser","browser":{}}]}}} to class io.github.sashirestela.openai.domain.assistant.ThreadRunStepDelta.
at io.github.sashirestela.cleverclient.util.JsonUtil.jsonToObject(JsonUtil.java:55)
at io.github.sashirestela.cleverclient.client.HttpClientAdapter.lambda$convertToStreamOfEvents$3(HttpClientAdapter.java:158)
at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:179)
at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133)
at java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1939)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
at ai.persona.services.openai.OpenaiCommunicationService.handleRunEvents(OpenaiCommunicationService.java:674)
at ai.persona.services.openai.OpenaiCommunicationService.runConversation(OpenaiCommunicationService.java:354)
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:354)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:196)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:768)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:768)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:720)
at ai.persona.services.openai.OpenaiCommunicationService$$SpringCGLIB$$0.runConversation()
at ai.persona.services.ContextRequestsHandleService.processRequest(ContextRequestsHandleService.java:105)
at ai.persona.services.events.ContextRequestEventsAsyncListener.handleContextRequestEvent(ContextRequestEventsAsyncListener.java:24)
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:354)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:196)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:768)
at org.springframework.aop.interceptor.AsyncExecutionInterceptor.lambda$invoke$0(AsyncExecutionInterceptor.java:113)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
at java.base/java.lang.Thread.run(Thread.java:1583)
Caused by: com.fasterxml.jackson.databind.exc.InvalidTypeIdException: Could not resolve type id 'browser' as a subtype of
io.github.sashirestela.openai.domain.assistant.StepDetail$ToolCallsStep$StepToolCall: known type ids = [code_interpreter, file_search, function] (for POJO property 'tool_calls')The text was updated successfully, but these errors were encountered: