[https://nvbugs/6611817][test] Waive known-flaky disagg + Nemotron tests on DGX_H100 - #17712
Conversation
…sts on DGX_H100
Three tests flake on main CI (stage DGX_H100-4_GPUs-PyTorch-Others-2),
independent of any branch, so SKIP-waive them with their tracking bugs:
- accuracy/test_disaggregated_serving.py::TestLlama3_1_8BInstruct::test_tp_pp_symmetric[MMLU-tp2pp1]
disagg-server timeout/hang, ~1.3% over 113 MRs -> nvbugs/6611817
- disaggregated/test_auto_scaling.py::test_disagg_server_restart[etcd-round_robin]
co-fails on the same stage/runs, ~1.2% over 112 MRs -> nvbugs/6611817 (shared)
- accuracy/test_llm_api_pytorch.py::TestNemotronV3Super::test_nvfp4_4gpus_hopper_w4a16
nvfp4 Hopper w4a16 flake -> nvbugs/6478723 (existing, TRTLLM-14447)
Pulled out of PR NVIDIA#17482 into a standalone waives-only PR so they land fast
with real bug references instead of placeholders.
Signed-off-by: Brian Nguyen <brnguyen@nvidia.com>
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: 📒 Files selected for processing (1)
WalkthroughThree integration tests are added to the waiver list. Each entry references NVBug 6611817. ChangesIntegration test waivers
Estimated code review effort: 1 (Trivial) | ~2 minutes Merge Risk: ⚪ Minimal · up to This PR only adds three known-flaky test waivers and does not change product or runtime code; no actionable merge-blocking risk remains after normal checks and review. Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
|
/bot skip --comment "waives only PR" |
|
PR_Github #66322 [ skip ] triggered by Bot. Commit: |
|
PR_Github #66322 [ skip ] completed with state |
Description
Standalone waives-only PR: SKIP-waive three tests that flake on
mainCI,each with a real tracking bug. These were pulled out of #17482 so they land
fast with real bug references instead of the placeholder-then-amend flow.
All three flake on stage
DGX_H100-4_GPUs-PyTorch-Others-2, independent of anyin-flight branch (they fail across 110+ unrelated MRs and multiple users on main):
test_disaggregated_serving.py::TestLlama3_1_8BInstruct::test_tp_pp_symmetric[MMLU-tp2pp1]test_auto_scaling.py::test_disagg_server_restart[etcd-round_robin]test_llm_api_pytorch.py::TestNemotronV3Super::test_nvfp4_4gpus_hopper_w4a16The first two co-fail on the same stage in the same runs — a shared disagg
server timeout/hang — so they share one new tracking bug (6611817). The
Nemotron nvfp4 Hopper flake folds into its existing open tracking bug.
Test Coverage
waives-only change (no source). Validated by the pre-commit hooks that ran on
commit: "Validate test list entries exist in source files (AST)", "Checks for
duplicated test items in waives.txt", and the sort/format checks — all passed.
PR Checklist
Dev Engineer Review
tests/integration/test_lists/waives.txt.nvbugs/6611817.nvbugs/6478723(TRTLLM-14447).Verdict: sufficient.
QA Engineer Review
tests/integration/test_lists/waives.txtchanged.test-db/orqa/files were modified.Verdict: needs follow-up.