[None][fix] Forward reasoning_effort to the chat template

[joerowell]

Dev Engineer Review

• reasoning_effort forwards to chat templates.
• Explicit request values override chat_template_kwargs.
• Unset values are omitted.
• reasoning_effort accepts none, minimal, xhigh, and max.
• Harmony maps max and xhigh to high reasoning.
• Unsupported values return None instead of raising KeyError.
• The changes preserve request immutability and existing template kwargs.
• No configuration or test-list files changed.

QA Engineer Review

Added CPU-only tests:

• test_a_sent_level_reaches_the_template_unchanged
• test_an_unsent_effort_is_not_forwarded
• test_a_sent_level_overrides_chat_template_kwargs
• test_chat_template_kwargs_still_work_on_their_own
• test_other_template_kwargs_are_preserved
• test_the_request_kwargs_are_not_mutated
• test_harmony_tolerates_every_level_the_field_accepts
• test_harmony_still_maps_the_levels_it_owns

No corresponding tests/integration/test_lists/, test-db/, or qa/ coverage entry was found. Verdict: needs follow-up.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: b04f187b-09a2-468f-a60a-48befb20ec96

📥 Commits

Reviewing files that changed from the base of the PR and between 0d55bbc and 0856a36.

📒 Files selected for processing (1)

• tensorrt_llm/serve/harmony_adapter.py

🚧 Files skipped from review as they are similar to previous changes (1)

• tensorrt_llm/serve/harmony_adapter.py

---

Walkthrough

The change expands accepted reasoning-effort values, forwards explicit values to chat templates, preserves existing template kwargs, and prevents unsupported Harmony values from raising KeyError. CPU-only tests cover forwarding and Harmony behavior.

Changes

Reasoning effort support

Layer / File(s)	Summary
Reasoning effort contracts and Harmony handling tensorrt_llm/serve/openai_protocol.py, tensorrt_llm/serve/harmony_adapter.py	The request accepts none, minimal, xhigh, and max. Harmony maps xhigh and max to high reasoning and returns None for unsupported values.
Template kwargs forwarding tensorrt_llm/serve/openai_server.py	Chat completion rendering copies chat_template_kwargs, applies explicit reasoning_effort, and preserves other kwargs.
Forwarding and Harmony validation tests/unittest/llmapi/apps/test_reasoning_effort_resolution.py	CPU-only tests cover forwarding, precedence, omission, immutability, and Harmony mappings.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Mergeability Score: ⚪ Minimal · up to 0856a

This change forwards reasoning_effort through chat-template handling, with no identified concrete correctness, security, availability, deployment, or permission risk at the current head; it is merge-ready after normal checks.

Sequence Diagram(s)

sequenceDiagram
  participant Client
  participant ChatCompletionRequest
  participant OpenAIServer
  participant ChatTemplate
  Client->>ChatCompletionRequest: send reasoning_effort
  ChatCompletionRequest->>OpenAIServer: request and chat_template_kwargs
  OpenAIServer->>OpenAIServer: merge explicit reasoning_effort
  OpenAIServer->>ChatTemplate: render with merged kwargs

Loading

Suggested reviewers: mikeiovine, allisonlim-nv

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name	Status	Explanation	Resolution
Description check	⚠️ Warning	No pull request description was provided, so the required issue, solution, test coverage, and checklist information is missing.	Add a description that explains the issue and solution, lists relevant tests, and addresses the required checklist items.
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (3 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title follows the required format and clearly identifies forwarding reasoning_effort to the chat template.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 2

🧹 Nitpick comments (1)

tensorrt_llm/serve/openai_server.py (1)

208-232: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Add a precise request type annotation.

Annotate request as ChatCompletionRequest. This helper reads Pydantic request fields and is part of the chat-template contract.

Proposed fix

-def _chat_template_kwargs_with_effort(request) -> dict:
+def _chat_template_kwargs_with_effort(
+        request: ChatCompletionRequest) -> dict[str, Any]:

As per coding guidelines, “Annotate every function” and “use precise types instead of dict/object/Any where applicable.”

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tensorrt_llm/serve/openai_server.py` around lines 208 - 232, Update
_chat_template_kwargs_with_effort to annotate request as ChatCompletionRequest,
reusing the existing import or adding the appropriate import if needed. Keep the
current return annotation and behavior unchanged.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tensorrt_llm/serve/harmony_adapter.py`:
- Around line 1922-1929: Add an "xhigh" entry to the str_to_effort mapping used
by reasoning-effort resolution, mapping it to ReasoningEffort.HIGH alongside
"max", so both requests resolve identically while preserving existing fallback
behavior.

In `@tests/unittest/llmapi/apps/test_reasoning_effort_resolution.py`:
- Around line 32-104: Run the repository-configured YAPF and ruff formatters on
the test file containing make_request, _chat_template_kwargs_with_effort, and
maybe_transform_reasoning_effort, then commit all formatter-generated changes
without altering the test behavior.

---

Nitpick comments:
In `@tensorrt_llm/serve/openai_server.py`:
- Around line 208-232: Update _chat_template_kwargs_with_effort to annotate
request as ChatCompletionRequest, reusing the existing import or adding the
appropriate import if needed. Keep the current return annotation and behavior
unchanged.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 184d25ea-16fd-4cd1-b28c-a42e51c2cffc

📥 Commits

Reviewing files that changed from the base of the PR and between 07b3e82 and 601faaf.

📒 Files selected for processing (4)

• tensorrt_llm/serve/harmony_adapter.py
• tensorrt_llm/serve/openai_protocol.py
• tensorrt_llm/serve/openai_server.py
• tests/unittest/llmapi/apps/test_reasoning_effort_resolution.py