[https://nvbugs/6287721][chore] Unwaive Qwen3 FP8 weight-update tests

[shikicloud]

Summary by CodeRabbit

• Removed the part0 and specific FP8 Qwen3 8B waivers.
• Retained the multi-GPU part1 and part4 waivers.
• Replaced the 4-GPU Ray part0 marker with explicit FP8 weight-update tests for Qwen3 8B and Qwen3 30B-A3B.

Dev Engineer Review

• The changes use valid test-list formats.
• The explicit test entries match the intended Qwen3 FP8 weight-update coverage.
• No code, API, performance, or error-handling changes are included.
• No duplicate or unintended public test entries were identified.

QA Engineer Review

• Modified tests/integration/test_lists/waives.txt:
  • Removed the part0 waiver.
  • Removed the specific FP8 Qwen3 8B waiver.
  • Retained the part1 and part4 waivers.
• Modified tests/integration/test_lists/test-db/l0_dgx_b200.yml:
  • Removed the part0 marker-based entry.
  • Added explicit FP8 weight-update tests for Qwen3 8B and Qwen3 30B-A3B.
• CI confirmed that the unwaived tests passed.
• Verdict: sufficient.

Description

PR #16700 waived the Ray orchestrator part0 update-weight tests under NVBug 6490036 after CI flakiness that could not be reproduced locally.

This PR restores the Qwen3 FP8 update-weight coverage by listing the two current part0 parameterizations as explicit test IDs:

Qwen3 8B → Qwen3 8B FP8

Qwen3 30B-A3B → Qwen3 30B-A3B FP8

Using explicit test IDs allows the two cases to be scheduled, diagnosed, and waived independently instead of running them through the shared part0 marker selection. The existing part1 and part4 waivers remain unchanged.

Test Coverage

Full pre-merge CI passed for commit 0f87f28, including the Multi-GPU test stage: L0_MergeRequest_PR #53050.

No product-code fix is included in this PR. The previously waived Qwen3 8B and Qwen3 30B-A3B FP8 cases now pass in pre-merge CI, so this PR restores their coverage using explicit test IDs.

PR Checklist

Please review the following before submitting your PR:

• PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

• PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

• Test cases are provided for new code paths (see test instructions)

• If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.

• Any new dependencies have been scanned for license and vulnerabilities

• CODEOWNERS updated if ownership changes

• Documentation updated as needed

• Update tava architecture diagram if there is a significant design change in PR.

• The reviewers assigned automatically/manually are appropriate for the PR.

• Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

[shikicloud]

/bot run --stage-list "DGX_B200-4_GPUs-PyTorch-Ray-1"

[coderabbitai]

Walkthrough

The Ray 4-GPU pre-merge list now runs explicit FP8 weight-update tests for Qwen3 8B and Qwen3 30B-A3B. The obsolete part0 and Qwen3 8B waivers were removed.

Changes

Ray FP8 test selection

Layer / File(s)	Summary
Configure explicit FP8 tests and waivers tests/integration/test_lists/test-db/l0_dgx_b200.yml, tests/integration/test_lists/waives.txt	The test list replaces the part0 marker with explicit Qwen3 8B and Qwen3 30B-A3B FP8 weight-update tests. The waiver list removes the matching part0 and Qwen3 8B entries while retaining part1 and part4 waivers.

Estimated code review effort: 1 (Trivial) | ~2 minutes

Suggested reviewers: qijune, brnguyen2, pcastonguay, bowenfu

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title follows the required format and clearly describes restoring Qwen3 FP8 weight-update test coverage.
Description check	✅ Passed	The description explains the issue, solution, test coverage, and checklist status with sufficient detail.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[tensorrt-cicd]

PR_Github #58151 [ run ] triggered by Bot. Commit: 2e0fedd Link to invocation

[Superjomn]

LGTM

[shikicloud]

/bot run --stage-list "DGX_B200-4_GPUs-PyTorch-Ray-1"

[tensorrt-cicd]

PR_Github #58176 [ run ] triggered by Bot. Commit: a31e9e0 Link to invocation

[tensorrt-cicd]

PR_Github #58151 [ run ] completed with state ABORTED. Commit: 2e0fedd

Link to invocation

[tensorrt-cicd]

PR_Github #58176 [ run ] completed with state FAILURE. Commit: a31e9e0
/LLM/main/L0_MergeRequest_PR pipeline #46826 (Partly Tested) completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[shikicloud]

/bot run --stage-list "DGX_B200-4_GPUs-PyTorch-Ray-1"

[shikicloud]

/bot run --stage-list "DGX_B200-4_GPUs-PyTorch-Ray-1"

[tensorrt-cicd]

PR_Github #58252 [ run ] triggered by Bot. Commit: ad85689 Link to invocation

[tensorrt-cicd]

PR_Github #58252 [ run ] completed with state FAILURE. Commit: ad85689
/LLM/main/L0_MergeRequest_PR pipeline #46893 (Partly Tested) completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[shikicloud]

/bot run --stage-list "DGX_B200-4_GPUs-PyTorch-Ray-1"

[tensorrt-cicd]

PR_Github #58267 [ run ] triggered by Bot. Commit: 3f55b11 Link to invocation

[tensorrt-cicd]

PR_Github #58267 [ run ] completed with state SUCCESS. Commit: 3f55b11
/LLM/main/L0_MergeRequest_PR pipeline #46906 (Partly Tested) completed with status: 'SUCCESS'

CI Report

Link to invocation

[Superjomn]

The unwaived tests have passed: https://prod.blsm.nvidia.com/sw-tensorrt-llm-github-2/blue/organizations/jenkins/LLM%2Fmain%2FL0_Test-x86_64-Multi-GPU/detail/L0_Test-x86_64-Multi-GPU/2273/tests

[Superjomn]

/bot skip --comment "the unwaived tests passed"

[tensorrt-cicd]

PR_Github #58929 [ skip ] triggered by Bot. Commit: 3f55b11 Link to invocation

[tensorrt-cicd]

PR_Github #58929 [ skip ] completed with state SUCCESS. Commit: 3f55b11
Skipping testing for commit 3f55b11

Link to invocation

[shikicloud]

/bot run --stage-list "DGX_B200-4_GPUs-PyTorch-Ray-1" -- unittest/_torch/ray_orchestrator/multi_gpu/test_llm_update_weights_multi_gpu.py::test_llm_update_weights_fp8[Qwen3/Qwen3-8B-Qwen3/Qwen3-8B-FP8] --run-ray

[coderabbitai]

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.

[shikicloud]

/bot run

[tensorrt-cicd]

PR_Github #65243 [ run ] triggered by Bot. Commit: 0f87f28 Link to invocation

[tensorrt-cicd]

PR_Github #65243 [ run ] completed with state FAILURE. Commit: 0f87f28
/LLM/main/L0_MergeRequest_PR pipeline #53021 completed with status: 'UNSTABLE'

CI Report

⚠️ Multi-GPU Label Required:
Multi-GPU tests require the ci: full pre-merge approved label on this PR. Ask a member of NVIDIA/trt-llm-ci-approvers to add the label, then re-trigger CI with the same bot command (no rebase needed).

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation

[shikicloud]

/bot run

[tensorrt-cicd]

PR_Github #65261 [ run ] triggered by Bot. Commit: 0f87f28 Link to invocation

[tensorrt-cicd]

PR_Github #65261 [ run ] completed with state FAILURE. Commit: 0f87f28
/LLM/main/L0_MergeRequest_PR pipeline #53038 completed with status: 'UNSTABLE'

CI Report

⚠️ Multi-GPU Label Required:
Multi-GPU tests require the ci: full pre-merge approved label on this PR. Ask a member of NVIDIA/trt-llm-ci-approvers to add the label, then re-trigger CI with the same bot command (no rebase needed).

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation

[shikicloud]

/bot run

[tensorrt-cicd]

PR_Github #65273 [ run ] triggered by Bot. Commit: 0f87f28 Link to invocation

[brnguyen2]

Approving — the comments below are optional touch-ups, not blockers.

Unwaive itself looks right: part0 currently contains exactly test_llm_update_weights_fp8[...8B] and [...30B-A3B], so the two explicit entries reproduce the old marker's coverage, and the part1/part4 waivers correctly stay.

Two things worth adding to the description: what actually fixed https://nvbugs/6287721 (or a link to the CI run that shows the two tests passing), and why the marker entry was replaced with explicit IDs rather than just dropping the waiver line.

[tensorrt-cicd]

PR_Github #65273 [ run ] completed with state SUCCESS. Commit: 0f87f28
/LLM/main/L0_MergeRequest_PR pipeline #53050 completed with status: 'SUCCESS'

CI Report

Link to invocation