Skip to content

Pull requests: EleutherAI/lm-evaluation-harness

New pull request New
146 Open 1,542 Closed
146 Open 1,542 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Reviews
Filter by reviews
No reviews Review required Approved review Changes requested
Assignee
Filter by who’s assigned
Assigned to nobody Loading
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Pull requests list

Add DETAILS.md for improved documentation
#3141 opened Jul 12, 2025 by ginylil-tech Loading…
1
Removed repeated "Let''s think step by step." text from bbh cot prompts
#3140 opened Jul 12, 2025 by philipdoldo Loading…
1
Added mixed_precision_dtype argument to HFLM to enable autocasting
#3138 opened Jul 11, 2025 by Avelina9X Loading…
Fix mmlu_continuation subgroup names to fit Readme and other variants
#3137 opened Jul 11, 2025 by lamalunderscore Loading…
Fix for hang due to mp.Pool in bootstrap_stderr
#3135 opened Jul 11, 2025 by ankitgola005 Loading…
1
Add tasklist
#3133 opened Jul 11, 2025 by baberabb Loading…
when using vllm with lora, it will have some mistakes, now i fix it.
#3132 opened Jul 11, 2025 by Jacky-MYQ Loading…
1
FixBug: Fix the wrong configs for gpqa_cot_n_shot
#3131 opened Jul 11, 2025 by Summer-Summer Loading…
1
Fix: extended to max_gen_toks 8192 for HRM8K math benchmarks
#3124 opened Jul 10, 2025 by shing100 Loading…
1
Update README.md for mlqa
#3117 opened Jul 7, 2025 by newme616 Loading…
1
set repeat metrics from config
#3109 opened Jul 5, 2025 by baberabb Loading…
add metric configs
#3105 opened Jul 4, 2025 by baberabb Loading…
Add support for OpenVINO text2text generation models
#3101 opened Jul 3, 2025 by nikita-savelyevv • Draft
2
Adds Anthropic/discrim-eval to lm-evaluation-harness
#3091 opened Jun 27, 2025 by Helw150 Loading…
1
2
fix request hanging when request api
#3090 opened Jun 27, 2025 by mmmans Loading…
3
feat(api_models): add enable_thinking param in chat_template_kwargs
#3088 opened Jun 27, 2025 by johnsonafool Loading…
1
add strip_thinking param
#3087 opened Jun 26, 2025 by baberabb Loading…
Refactor ConfigurableTask.process_results into modular helpers
#3085 opened Jun 25, 2025 by mfisher35 Loading…
2
[Proposal] Change hyphens in n-shot and n-samples to underscores
#3084 opened Jun 24, 2025 by kiersten-stokes Loading…
Fix Typo in Answer Explanation
#3071 opened Jun 19, 2025 by leopardracer Loading…
2
Custom request headers | trust_remote_code param fix
#3069 opened Jun 18, 2025 by RawthiL Loading…
3
improve include-path precedence handling
#3068 opened Jun 18, 2025 by parkhs21 Loading…
1
3
Gracefully skip BigBench tasks with no data & guard final aggregation
#3066 opened Jun 17, 2025 by NourFahmy Loading…
2
Sync with base repository
#3064 opened Jun 17, 2025 by OliverSkola Loading…
1
Adding EgyMMLU and EgyHellaSwag
#3063 opened Jun 16, 2025 by houdaipha Loading…
2 of 6 tasks
3
ProTip! Type g p on any issue or pull request to go back to the pull request listing page.