What is the complete inference process?

[zhiyuan5986]

Hi, this is a great job!

My question is: In the paper's `Inference' part, you introduce the process to generate $C_{\text{compressed}}$. Then which model are used in the following step (i.e. get the output answer based on query and C_\text{compressed}$)?

Thanks a lot!

[No13Judas]

Refer to the script mentioned in the README, I guess it's gpt-3.5-turbo, and I get 92% of the longbench avg score which was shown in the paper using the model, also they showed the scores with gpt-4o in the paper.