Skip to content

moses607/adaptive-eval

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

1 Commit
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

📏 adaptive-eval

Rank LLMs with a fraction of the benchmark items — same ranking, a lot less compute.

Running a full benchmark (thousands of items) for every model and every checkpoint is slow and expensive, and most of those items are uninformative — everyone passes them or everyone fails them, so they cost tokens and add no signal. adaptive-eval borrows Computerized Adaptive Testing (the method behind the GRE): model each item with a difficulty and a discrimination, then administer only the items that carry the most Fisher information about this model's ability, and stop as soon as the ranking is pinned.

Small, readable, zero dependencies (pure Python). A drop-in library, not a framework.


The result (run it yourself)

python -m adaptive_eval.sim
item bank: 300 items | models ranked: 40
adaptive used on average 13.9 items (4.7% of the bank)
ranking fidelity vs full bank (Spearman):
  ADAPTIVE : 0.949
  RANDOM (same budget): 0.843

On this synthetic bank, adaptive testing reproduces 95% of the full-bank ranking using under 5% of the items, and clearly beats random subsampling at the same budget.

Honest scope: this is a synthetic, well-specified demonstration (responses are generated from the same one-dimensional IRT model the estimator assumes). Real benchmarks are multi-skill and noisier, so expect to spend more items and to need good calibration. This proves the mechanism; it is not a claim about any specific real benchmark. See Limits.


Install & use

git clone https://github.com/moses607/adaptive-eval && cd adaptive-eval
pip install -e .
from adaptive_eval import calibrate, adaptive_test

# 1) Calibrate item params ONCE from models you already ran in full (0/1 matrix).
items = calibrate(response_matrix)      # -> [(discrimination, difficulty), ...]

# 2) Evaluate a NEW model adaptively. answer_fn(i) runs item i, returns 1/0.
result = adaptive_test(items, answer_fn, se_threshold=0.33)
print(result.theta, result.se, result.n_items)   # ability, error, items used

See examples/quickstart.py.

How it works (the whole idea, in three lines)

  • Item Response Theory (2PL): p(correct) = 1 / (1 + e^-a(θ − b)) — item difficulty b, discrimination a, model ability θ.
  • Fisher information: an item tells you most about θ when its difficulty sits near θI = a²·p·(1−p).
  • Adaptive loop: administer the max-information item, re-estimate θ, stop when the standard error is small enough. The model never sees the whole bank.

The math lives in adaptive_eval/core.py (~120 lines) and adaptive_eval/adaptive.py. Read it in one sitting.

Honest novelty (what this is — and isn't)

The idea is not new, and usable tools already exist. Be clear-eyed:

  • Research: ATLAS — Adaptive Testing for LLM Evaluation (2025, Fisher-info selection, up to 90% fewer items — arXiv:2511.04689) and Confident Rankings with Fewer Items (2026 — arXiv:2601.13885).
  • Tools: CAT4AI (a full adaptive-testing framework for AI models), adaptivetesting (a broad Bayesian CAT package), tinyBenchmarks (pre-selected static subsets + an IRT tool), plus general CAT libs (catsim, mirtCAT).

So why adaptive-eval? Minimalism. It's the smallest version — ~120 readable lines, zero dependencies, a two-function drop-in (calibrate, adaptive_test) you can read in one sitting and vendor straight into your own eval loop, selecting items adaptively per run (not a fixed pre-chosen subset). Want a full framework? Use CAT4AI. Want the thing you can read, trust, and paste in? Use this. Think "the nanoGPT of adaptive LLM eval" — not "the first."

Limits (read before you trust it)

  • One-dimensional ability. Real benchmarks are multi-skill; a single θ blurs that. Multidimensional IRT is on the roadmap.
  • Needs calibration. Adaptive selection needs item parameters. calibrate() is a solid JMLE baseline (recovers item difficulty at Spearman ≈ 0.99 on synthetic data) — supply your own for production.
  • Assumes stable items. If an item's behavior drifts, its calibration goes stale.
  • The demo is idealized. See the scope note above.

Roadmap

Multidimensional (per-skill) ability · an adapter for lm-eval-harness / HELM result matrices · continuous-score items (not just right/wrong) · exposure control so you don't overuse the same items.

License

MIT © 2026 Cherry FRANCOIS. From the maker of Aether OS — whose kernel has an eval harness this extends.

About

Rank LLMs with a fraction of the benchmark items, same ranking — adaptive (IRT/CAT) evaluation as a small, zero-dependency, drop-in Python library. The nanoGPT of adaptive LLM eval.

Topics

Resources

License

Stars

0 stars

Watchers

0 watching

Forks

Releases

No releases published

Packages

 
 
 

Contributors

Languages