Skip to content

[https://nvbugs/6604925][test] Waive flaky TestQwen3_6_35B_A3B::test_nvfp4[TRTLLM] (SIGSEGV on DGX_B200) - #17640

Merged
brnguyen2 merged 1 commit into
NVIDIA:mainfrom
brnguyen2:user/brnguyen/waive-qwen3-6-nvfp4-segfault
Aug 13, 2026
Merged

[https://nvbugs/6604925][test] Waive flaky TestQwen3_6_35B_A3B::test_nvfp4[TRTLLM] (SIGSEGV on DGX_B200)#17640
brnguyen2 merged 1 commit into
NVIDIA:mainfrom
brnguyen2:user/brnguyen/waive-qwen3-6-nvfp4-segfault

Conversation

@brnguyen2

@brnguyen2 brnguyen2 commented Aug 13, 2026

Copy link
Copy Markdown
Collaborator

What

Waive accuracy/test_llm_api_pytorch.py::TestQwen3_6_35B_A3B::test_nvfp4[TRTLLM] — NVBug https://nvbugs/6604925.

Why

The test intermittently crashes the pytest process with SIGSEGV (exit 139) on DGX_B200 — a real in-process crash, not an assertion or an infra timeout (the infra classifier matched no infra signature). Flake history over the last 14 days: ~1% (9/885 runs) across ~330 unrelated MRs and 5 distinct users, all on DGX_B200 — a main-side flake, not PR-correlated. Fail-fast turns each occurrence into a red pipeline, so it's worth waiving while the crash is root-caused under the NVBug.

Scope (DGX_B200, implicitly)

nvfp4 is Blackwell-only, so this test appears only in B200 test lists (test-db/l0_b200.yml, qa/llm_function_core.txt) and never runs on non-B200 platforms. A plain waives.txt entry (no full:<platform>/ prefix) is therefore already scoped to DGX_B200 by construction — and unlike a full:B200/-prefixed entry, a plain entry reliably applies in the L0 context (DGX_B200-PyTorch-3 stage) where the crash was observed. Only the [TRTLLM] variant is waived; [CUTEDSL] is left running.

Tracked by https://nvbugs/6604925.

Dev Engineer Review

  • Added one waiver entry for accuracy/test_llm_api_pytorch.py::TestQwen3_6_35B_A3B::test_nvfp4[TRTLLM].
  • The entry references NVBug 6604925.
  • The waiver scope is limited to the [TRTLLM] variant.
  • The change uses the expected waives.txt format and does not alter public entities.
  • The waiver is consistent with the reported intermittent SIGSEGV on DGX_B200.

QA Engineer Review

  • Modified files: tests/integration/test_lists/waives.txt.
  • Added one waiver entry.
  • No test code changed.
  • CBTS coverage data is unavailable. Verdict: needs follow-up.

…nvfp4[TRTLLM] (SIGSEGV on DGX_B200)

test_nvfp4[TRTLLM] intermittently crashes the pytest process with SIGSEGV
(exit 139) on DGX_B200 (~1% over a 14-day window, across many unrelated MRs
and users). nvfp4 is Blackwell-only, so this test runs only on B200 test
lists (l0_b200.yml, qa/llm_function_core.txt); the plain waive is therefore
B200-scoped by construction. Tracked by https://nvbugs/6604925.

Signed-off-by: Brian Nguyen <brnguyen@nvidia.com>
@brnguyen2

Copy link
Copy Markdown
Collaborator Author

/bot run

@coderabbitai

coderabbitai Bot commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 72ec1e19-ad64-4cf7-84e0-858b2e91ff07

📥 Commits

Reviewing files that changed from the base of the PR and between 7647daa and 6b99b34.

📒 Files selected for processing (1)
  • tests/integration/test_lists/waives.txt

Walkthrough

The integration waiver list adds an NVBug-linked skip for the Qwen3.6 35B A3B NVFP4 TRTLLM accuracy test.

Changes

Qwen3.6 NVFP4 test waiver

Layer / File(s) Summary
Add NVFP4 accuracy test skip
tests/integration/test_lists/waives.txt
Adds a SKIP entry for TestQwen3_6_35B_A3B::test_nvfp4[TRTLLM] with NVBug 6604925.

Estimated code review effort: 1 (Trivial) | ~2 minutes

Mergeability Score: ⚪ Minimal · up to 6b99b

This change only adds a narrowly scoped test waiver, and no actionable merge-blocking risk remains after normal checks and review.

Possibly related PRs

Suggested reviewers: qijune

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly identifies the NVBug, waiver, affected test variant, failure mode, and DGX_B200 platform.
Description check ✅ Passed The description explains what changed, why the waiver is needed, the failure history, platform scope, and the retained test variant.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #65923 [ run ] triggered by Bot. Commit: 6b99b34 Link to invocation

@brnguyen2

Copy link
Copy Markdown
Collaborator Author

/bot skip --comment "waive-only PR"

@brnguyen2
brnguyen2 enabled auto-merge (squash) August 13, 2026 13:34
@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #65924 [ skip ] triggered by Bot. Commit: 6b99b34 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #65923 [ run ] completed with state ABORTED. Commit: 6b99b34

Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #65924 [ skip ] completed with state SUCCESS. Commit: 6b99b34
Skipping testing for commit 6b99b34

Link to invocation

@brnguyen2
brnguyen2 merged commit 265a116 into NVIDIA:main Aug 13, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants