The annotation type consistently returns as 'file_citation' even when a web search is performed with the Web Search toggle turned off

[casey55]

I've uploaded a file to the vector store and have enabled only "File Search," while "Web Search" and "Functions" are unchecked. When I ask questions directly related to the content of my uploaded file, the citation correctly points to it. My uploads contain information about accounting software.

However, when I ask an unrelated question, such as "Who won the Super Bowl in 2020?", I expected no results. Instead, I received the correct answer, but the citation incorrectly references my uploaded file.

My assumption was that with "Web Search" disabled, only my uploaded files would be searched. I did notice a message stating, "If they need up-to-date information, you can use the web search tool to search the web for relevant information." This seems to explain why a web search is still being performed, but it doesn't explain why my uploaded file is cited for the web search results.

Interestingly, if I enable "Web Search," the citations report correctly. Is this the expected behavior when "Web Search" is disabled?

[katia-openai]

Hi @casey55, not sure I fully understand what the problem is here.

If the model only has access to file search, it will only use file search and it's possible that in this instance it used the tool even when it was unnecessary because a question like "Who won the Super Bowl in 2020?" would be part of the model's training data and therefore inherent knowledge. If the tool is used and you have one file in your vector store, the only thing it can reference is that file and you would see a file citation.

From what you're describing, it doesn't sound like web search is being used, which is the expected behaviour when web search is disabled.

[casey55]

Hello @katia-openai Thanks for replying.

So, you're saying that I will get an answer to the Super Bowl question even though my uploaded file is totally unrelated, because the AI inherently knows the answer? I think I see my misunderstanding: web search is unrelated to the AI's current inherent knowledge.

1. Is there a way to only get answers from my uploaded document?
2. The AI returning an answer isn't so much a problem, but its citing my document as the source of the answer is a problem.

Thanks,
kc

[katia-openai]

Yes, if you want to only get answers from the uploaded documents, you can try prompting the model (you can change the prompt in config/constants.ts) with something like "Only use the file_search tool to answer when the user is asking a knowledge question, do not answer based on your own knowledge".

It is indeed unexpected that the model would cite your document for no reason if it hasn't used the knowledge in your document. An example would be helpful here to understand what's happening.