[https://nvbugs/6490036][test] Isolate part1 FP8 weight-update tests - #17503
Conversation
Signed-off-by: Shiki Wu <shikiw@nvidia.com>
WalkthroughThe pre-merge configuration now selects explicit multi-GPU FP8 partial weight-update tests for Qwen3 8B and Qwen3 30B A3B. The waiver no longer skips the ChangesFP8 weight-update test coverage
Estimated code review effort: 1 (Trivial) | ~3 minutes Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches 💡 1🛠️ Fix failing CI checks 💡
🧪 Generate unit tests (beta)
Comment |
|
/bot run |
|
PR_Github #65303 [ run ] triggered by Bot. Commit: |
|
PR_Github #65303 [ run ] completed with state
|
|
/bot run |
|
PR_Github #65311 [ run ] triggered by Bot. Commit: |
|
PR_Github #65311 [ run ] completed with state
|
brnguyen2
left a comment
There was a problem hiding this comment.
Checked -m "part1" against the source: it marks only test_llm_partial_update_weights_fp8, and its two parametrize IDs match the new entries exactly, so coverage is unchanged. Same pattern as #16102. Fine to merge once pre-merge CI confirms both isolated cases pass.
|
/bot run |
|
PR_Github #65442 [ run ] triggered by Bot. Commit: |
|
PR_Github #65442 [ run ] completed with state |
|
/bot run |
|
PR_Github #65453 [ run ] triggered by Bot. Commit: |
|
PR_Github #65453 [ run ] completed with state |
Dev Engineer Review
tests/integration/test_lists/test-db/l0_dgx_b200.ymlwith explicit Qwen3 8B and Qwen3 30B-A3B FP8 partial weight-update test IDs.part1waiver fromtests/integration/test_lists/waives.txt.QA Engineer Review
tests/integration/test_lists/test-db/l0_dgx_b200.yml.tests/integration/test_lists/waives.txt.part1waiver.Description
PR #16700 waived the Ray orchestrator part0 and part1 FP8 weight-update tests under NVBug 6490036 after intermittent failures in the same CI stage.
PR #16102 restored the part0 coverage by replacing the marker-based entry with its two explicit parametrizations. Both isolated part0 cases passed in pre-merge CI.
This follow-up applies the same isolation to the remaining part1 tests:
Qwen3 8B → Qwen3 8B FP8 partial weight update
Qwen3 30B-A3B → Qwen3 30B-A3B FP8 partial weight update
This PR replaces the shared -m "part1" test-list entry with the two explicit test IDs and removes the corresponding NVBug 6490036 waiver. Listing them separately allows each parametrization to run as an independent CI test-list entry and to be diagnosed or waived independently.
No product code or test implementation is changed. This restores the remaining waived coverage and tests the hypothesis that separate invocations prevent Ray/CUDA IPC state from leaking between parametrized cases.
Test Coverage
Pre-merge CI: CI65453.
PR Checklist
Please review the following before submitting your PR:
PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.
PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.
Test cases are provided for new code paths (see test instructions)
If PR introduces API changes, an appropriate PR label is added - either
api-compatibleorapi-breaking. Forapi-breaking, includeBREAKINGin the PR title.Any new dependencies have been scanned for license and vulnerabilities
CODEOWNERS updated if ownership changes
Documentation updated as needed
Update tava architecture diagram if there is a significant design change in PR.
The reviewers assigned automatically/manually are appropriate for the PR.
Please check this after reviewing the above items as appropriate for this PR.
GitHub Bot Help
To see a list of available CI bot commands, please comment
/bot help.