weird....why the new version become worse????? #594

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Open

yanghu819 opened this issue Apr 11, 2025 · 1 comment

yanghu819 commented Apr 11, 2025

Model	AIME 2024 (🤗 LightEval)	AIME 2024 (DeepSeek Reported)
DeepSeek-R1-Distill-Qwen-1.5B	26.7	28.9
DeepSeek-R1-Distill-Qwen-7B	56.6	55.5
DeepSeek-R1-Distill-Qwen-14B	60.0	69.7
DeepSeek-R1-Distill-Qwen-32B	73.2	72.6
DeepSeek-R1-Distill-Llama-8B	43.3	50.4
DeepSeek-R1-Distill-Llama-70B	73.3	70.0
DeepSeek-R1-Distill-Qwen-1.5B	31.8	28.9
DeepSeek-R1-Distill-Qwen-7B	52.2	55.5
DeepSeek-R1-Distill-Qwen-14B	66.5	69.7
DeepSeek-R1-Distill-Qwen-32B	68.0	72.6
DeepSeek-R1-Distill-Llama-8B	43.9	41.7
DeepSeek-R1-Distill-Llama-70B	65.3	70.0

this is your newly updated version, why worse...

kiseliu commented Apr 23, 2025

Same question. 👀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment