-
Notifications
You must be signed in to change notification settings - Fork 513
Pull requests: huggingface/lighteval
Author
Label
Projects
Milestones
Reviews
Assignee
Sort
Pull requests list
Fix unreachable response_format default in JudgeLM
#1303
opened Jul 17, 2026 by
oraziooztas
Loading…
Fix litellm judge backend: send max_tokens as an int, not a 1-tuple
#1302
opened Jul 17, 2026 by
oraziooztas
Loading…
Fix PEFT example and base-model config resolution for adapter models
#1301
opened Jul 17, 2026 by
oraziooztas
Loading…
fix: correct stop_sequence -> stop_sequences typo in _continuous_greedy_until
#1300
opened Jul 17, 2026 by
RudrenduPaul
Loading…
Fix asdiv prompt truncating multi-character gold answers
#1298
opened Jul 14, 2026 by
vineethsaivs
Loading…
Fix max_tokens sent as a list by the litellm judge backend
#1297
opened Jul 14, 2026 by
AbhilakshSinghReen
Loading…
fix: MixEval judge extractors use last-match (verdict-injection defense)
#1295
opened Jul 13, 2026 by
AUTHENSOR
Loading…
4 tasks done
Use ast.literal_eval instead of eval in hellaswag_arabic_pfn
#1294
opened Jul 11, 2026 by
jayzuccarelli
Loading…
Fix string-name normalization in SamplingMetric (inspect.getmembers returns a list, not a dict)
#1292
opened Jul 8, 2026 by
teddytennant
Loading…
Add a regression test for truncated-then-full sample-cache runs
#1291
opened Jul 8, 2026 by
shipbehaves
Contributor
Loading…
fix(normalizations): exclude -1 padding from token-norm length + guard mismatch (#1170)
#1290
opened Jul 8, 2026 by
WatchTree-19
Loading…
Add Armenian language evaluation suite (ArmBench-LLM)
#1289
opened Jul 8, 2026 by
bugdaryan
Loading…
Add unit tests for VLLMModel sampling params and config wiring
#1287
opened Jul 7, 2026 by
jayzuccarelli
Loading…
Batch tokenization in the transformers backend's scoring loop
#1286
opened Jul 6, 2026 by
ErenAta16
Loading…
add missing docstrings to three public functions in utils/imports.py
#1282
opened Jul 4, 2026 by
RavSinghChandan
Loading…
Honor the model's generation_config.eos_token_id in the transformers backend
#1279
opened Jul 2, 2026 by
waqaskhan137
Loading…
Fix reversed gold index in boolq:contrastset prompt
#1277
opened Jun 29, 2026 by
vineethsaivs
Loading…
Fix CorpusLevelF1Score(None) to report positive-class F1, not max-per-class
#1275
opened Jun 25, 2026 by
iamsharduld
Loading…
Fix MajAtN IndexError when the gold is not the first choice
#1274
opened Jun 25, 2026 by
iamsharduld
Loading…
Fix helm_normalizer mis-scoring numbers (homogeneize before remove_punc)
#1273
opened Jun 25, 2026 by
iamsharduld
Loading…
Previous Next
ProTip!
Updated in the last three days: updated:>2026-07-14.