Fix litellm judge backend: send max_tokens as an int, not a 1-tuple

[oraziooztas]

Fixes #1296.

What does this PR do?

JudgeLM.__call_litellm builds the completion kwargs with a trailing comma:

kwargs["max_tokens"] = (max_new_tokens,)

The value serializes to a JSON array ("max_tokens": [512]), which OpenAI-compatible servers reject with a 400. After the retries, the judge returns the error string "ERROR: Failed to get response from the API." as if it were a judge response, so corrupted scores are produced without any crash or warning. This affects the litellm backend whenever max_tokens is set.

The fix sends the parameter as the integer it is:

kwargs["max_tokens"] = max_new_tokens

Tests

Added tests/unit/metrics/test_llm_as_judge.py with a stubbed litellm module (no network, no litellm install needed) that captures the kwargs passed to litellm.completion:

• test_litellm_max_tokens_is_sent_as_integer: fails on current main (captures (512,)), passes with the fix;
• test_litellm_omits_max_tokens_when_unset: max_tokens=None keeps the key out of the request.

Verification

• New tests: fail before the fix, 2/2 pass after
• tests/unit/metrics/: 288 passed; the single extractiveness.json failure is unrelated and reproduces identically on pristine main in the same environment
• ruff check and ruff format --check pass on both touched files

🤖 Generated with Claude Code

https://claude.ai/code/session_0158ZooCCJEy6xtme9c8iksT