[TRTLLM-15177][chore] Consolidate trtllm-gen SiTu activation slot handling by brnguyen2 · Pull Request #17414 · NVIDIA/TensorRT-LLM

brnguyen2 · 2026-08-07T08:48:17Z

Summary

Consolidate the trtllm-gen SiTu activation slot handling, a review follow-up from #17269 tracked in TRTLLM-15177 (item 1.2).

Included

Complete item 1.2(b): delete the separate situ_alpha/situ_beta parameters and reuse the swiglu_alpha/swiglu_beta storage. SiTu and SwiGLU are mutually exclusive and feed the same gemm1_alpha/gemm1_beta op slots, so the slot now has a single backing parameter and the interim _gemm1_activation_params accessor is gone.
Audited every swiglu_alpha is not None gate for behavior neutrality: create_moe.py validates the constructor kwargs (still None for SiTu; the storage is populated later in create_weights); _get_quant_method consults swiglu_alpha only on the nvfp4 branch (SiTu requires W4A8_MXFP4_MXFP8) and runs before the storage is populated; _check_configs's swiglu gate admits w4a8_mxfp4_mxfp8 and its SiTu branch validates the merged storage; the NVFP4 post-load swiglu_beta rescale is not in SiTu's MXFP4 quant-method ancestry; _validate_backend_local_activation runs from __init__ and still rejects constructor-provided SwiGLU parameters for SiTu.
Document, in _torch/utils.py, the deliberate decision to keep SiTu in the trtllm-gen-local ActType_TrtllmGen rather than adding it to the shared ActivationType. The shared enum mirrors the cutlass enum in common.h and drives cutlass kernels; SiTu exists only in the trtllm-gen batched-GEMM kernels, so adding it to the shared enum would force a cutlass member no kernel implements.

Validation

Run on a Blackwell (SM100-class) node:

tests/unittest/_torch/modules/moe/test_kimi_k3_situ_moe.py — full suite pass, including fused-vs-reference SiTu parity and MoE-TP shard-loader / TP8 sharded-forward parity.
tests/unittest/_torch/modules/moe/test_moe_backend.py -k trtllm — pass; covers the trtllm-gen SwiGLU leg including gptoss-style per-expert swiglu_alpha/swiglu_beta/swiglu_limit on W4A8_MXFP4_MXFP8, proving the merge did not regress the SwiGLU path.
tests/unittest/_torch/modules/moe/test_moe_module.py -k "trtllm and not multi_gpu" — pass (single-GPU ConfigurableMoE coverage). The DEP multi-GPU comm parametrizations are not runnable in this single-node harness (they need pre-spawned MPI ranks) and are covered by CI.
Kimi KDA spot checks: tests/unittest/_torch/modules/kimi_kda/ and tests/unittest/_torch/modeling/test_kimi_kda_{fused_,}verify_parity.py — pass.
Truncated Kimi K3 end-to-end sanity (4 layers, TP4, fused SiTu MoE path): load -> prefill -> decode -> shutdown, pass.

No enum values changed, so the Python/cutlass and Python/KernelRunner.h mappings are untouched.

Related: TRTLLM-15177

Dev Engineer Review

SiTu reuses swiglu_alpha and swiglu_beta for validation, weight creation, meta-device reinitialization, and MoE execution.
Separate SiTu storage and _gemm1_activation_params were removed.
SiTu and SwiGLU remain mutually exclusive and share GEMM1 activation slots.
Documentation explains the TRTLLM-Gen-specific ActType_TrtllmGen enum.
No enum values or mappings changed.
Two pending waivers reference PR 17414 for the PTP quickstart BERT end-to-end test and auto-deploy shim tests.
L0 merge-request pipelines repeatedly failed and require follow-up investigation.

QA Engineer Review

No test functions were added, modified, or removed.
tests/integration/test_lists/waives.txt adds two test-list entries.
No test-db/ or qa/ files were modified.
Verdict: needs follow-up because CBTS coverage data is unavailable.

Note: temporary CI-flake waives

The last commit adds three tests/integration/test_lists/waives.txt entries for DGX_B200 main-side flakes that gated this PR's CI — unrelated to the trtllm-gen SiTu change (each fails across hundreds of unrelated MRs and many users):

kv_cache/test_kv_cache_v2_scheduler.py::TestKVCacheV2Llama::test_chunked_prefill_eviction_block_reuse → https://nvbugs/6607481
unittest/_torch/modules/tests_lora_modules/test_qwen3_sanity.py::TestQwen3LoRA::test_qwen3_fp8_lora → https://nvbugs/6607487
unittest/_torch/modeling/test_gemma4_e2e_dummy.py::test_e2e_text_31b_dummy → https://nvbugs/6607482

These same waives are being landed on main independently in #17663. Once that merges, this branch rebases onto the waived main and these entries drop out — they're here only to get this PR green in the meantime.

brnguyen2 · 2026-08-07T14:08:42Z

/bot run

coderabbitai · 2026-08-07T14:13:42Z

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

@coderabbitai resume to resume automatic reviews.
@coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

▶️ Resume reviews
🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: f5dbcf13-f66d-4efa-abe0-8ce49f1c9596

📥 Commits

Reviewing files that changed from the base of the PR and between 9ace8cc and 66bb5e6.

📒 Files selected for processing (1)

tests/integration/test_lists/waives.txt

🚧 Files skipped from review as they are similar to previous changes (1)

tests/integration/test_lists/waives.txt

Walkthrough

TRTLLM-Gen fused MoE now stores SiTu activation parameters in shared SwiGLU tensors. Validation, initialization, and GEMM1 execution use this storage. Documentation and integration test waivers were updated.

Changes

SiTu parameter storage

Layer / File(s)	Summary
Shared activation parameter lifecycle `tensorrt_llm/_torch/modules/fused_moe/fused_moe_trtllm_gen.py`, `tensorrt_llm/_torch/utils.py`	SiTu uses non-trainable `swiglu_alpha` and `swiglu_beta` tensors for validation, initialization, meta-device materialization, and GEMM1 execution. Documentation records that SiTu remains specific to TRTLLM-Gen kernels.
Integration test waivers `tests/integration/test_lists/waives.txt`	Adds pending waivers for the PTP quickstart BERT test and auto-deploy shim tests.

Estimated code review effort: 2 (Simple) | ~10 minutes

Mergeability Score: ⚪ Minimal · up to 66bb5

This PR consolidates SiTu and SwiGLU activation-slot storage without changing enum values or intended behavior, and the supplied validation covers both paths; no actionable merge-blocking risk remains beyond normal checks and review.

Possibly related PRs

NVIDIA/TensorRT-LLM#17624: Changes overlapping TRTLLM-Gen SiTu activation parameter handling.
NVIDIA/TensorRT-LLM#17616: Updates related integration test waivers.

Suggested reviewers: barry-delaney, qijune, jieli-matrix

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description check	✅ Passed	The description clearly explains the SiTu storage consolidation, related design decisions, validation coverage, and temporary CI waives.
Title check	✅ Passed	The title follows the required format and clearly identifies the main change: consolidating TRTLLM-Gen SiTu activation slot handling.

✨ Finishing Touches 💡 1

🛠️ Fix failing CI checks 💡

Create stacked PR
Commit on current branch

🧪 Generate unit tests (beta)

Create PR with unit tests

_{Comment @coderabbitai help to get the list of available commands.}

tensorrt-cicd · 2026-08-07T14:15:18Z

PR_Github #64616 [ run ] triggered by Bot. Commit: e4e12fa Link to invocation

tensorrt-cicd · 2026-08-07T16:37:20Z

PR_Github #64616 [ run ] completed with state SUCCESS. Commit: e4e12fa
/LLM/main/L0_MergeRequest_PR pipeline #52476 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

brnguyen2 · 2026-08-07T16:41:47Z

/bot run

tensorrt-cicd · 2026-08-07T16:48:18Z

PR_Github #64651 [ run ] triggered by Bot. Commit: e4e12fa Link to invocation

tensorrt-cicd · 2026-08-07T18:03:40Z

PR_Github #64651 [ run ] completed with state SUCCESS. Commit: e4e12fa
/LLM/main/L0_MergeRequest_PR pipeline #52511 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

brnguyen2 · 2026-08-07T18:20:15Z

/bot run

tensorrt-cicd · 2026-08-07T18:26:21Z

PR_Github #64674 [ run ] triggered by Bot. Commit: be08d33 Link to invocation

tensorrt-cicd · 2026-08-07T18:55:15Z

PR_Github #64674 [ run ] completed with state FAILURE. Commit: be08d33
/LLM/main/L0_MergeRequest_PR pipeline #52531 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

brnguyen2 · 2026-08-07T19:09:57Z

/bot run

tensorrt-cicd · 2026-08-07T19:16:14Z

PR_Github #64683 [ run ] triggered by Bot. Commit: be08d33 Link to invocation

tensorrt-cicd · 2026-08-07T20:00:30Z

PR_Github #64683 [ run ] completed with state SUCCESS. Commit: be08d33
/LLM/main/L0_MergeRequest_PR pipeline #52539 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

brnguyen2 · 2026-08-07T20:46:22Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-08-07T20:51:55Z

PR_Github #64700 [ run ] triggered by Bot. Commit: be08d33 Link to invocation

tensorrt-cicd · 2026-08-08T00:26:52Z

PR_Github #64700 [ run ] completed with state SUCCESS. Commit: be08d33
/LLM/main/L0_MergeRequest_PR pipeline #52556 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

brnguyen2 · 2026-08-08T00:43:22Z

/bot run --disable-fail-fast --stage-list "DGX_B200-PyTorch-8, GH200-PackageSanityCheck-PY312-UB2404"

tensorrt-cicd · 2026-08-08T00:48:51Z

PR_Github #64733 [ run ] triggered by Bot. Commit: be08d33 Link to invocation

tensorrt-cicd · 2026-08-08T02:51:20Z

PR_Github #64733 [ run ] completed with state SUCCESS. Commit: be08d33
/LLM/main/L0_MergeRequest_PR pipeline #52584 (Partly Tested) completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

brnguyen2 · 2026-08-09T16:46:21Z

/bot run

tensorrt-cicd · 2026-08-13T17:20:39Z

PR_Github #65990 [ run ] triggered by Bot. Commit: 36c8df0 Link to invocation

…ut on RTXPro6000D-PyTorch-1, tracking bug pending) Signed-off-by: Brian Nguyen <brnguyen@nvidia.com>

brnguyen2 · 2026-08-13T18:40:42Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-08-13T18:50:00Z

PR_Github #66019 [ run ] triggered by Bot. Commit: 3b66f72 Link to invocation

tensorrt-cicd · 2026-08-13T18:52:40Z

PR_Github #65990 [ run ] completed with state ABORTED. Commit: 36c8df0

Link to invocation

These three tests flake on main CI independent of any change, failing across hundreds of unrelated MRs and many distinct users, with flake-typical signatures (hang/timeout, terminate-unexpectedly, assertion). Under fail-fast each flaky red gates the whole pipeline. Waive with tracking bugs: - kv_cache/test_kv_cache_v2_scheduler.py::TestKVCacheV2Llama::test_chunked_prefill_eviction_block_reuse (nvbugs/6607481) - unittest/_torch/modules/tests_lora_modules/test_qwen3_sanity.py::TestQwen3LoRA::test_qwen3_fp8_lora (nvbugs/6607487) - unittest/_torch/modeling/test_gemma4_e2e_dummy.py::test_e2e_text_31b_dummy (nvbugs/6607482) Signed-off-by: Brian Nguyen <brnguyen@nvidia.com>

brnguyen2 · 2026-08-13T22:31:25Z

/bot run --stage-list "DGX_B200-PyTorch-1,DGX_B200-PyTorch-6,DGX_B200-PyTorch-9"

tensorrt-cicd · 2026-08-13T22:38:02Z

PR_Github #66068 [ run ] triggered by Bot. Commit: 3392cd6 Link to invocation

tensorrt-cicd · 2026-08-13T22:41:15Z

PR_Github #66019 [ run ] completed with state ABORTED. Commit: 3b66f72

Link to invocation

tensorrt-cicd · 2026-08-14T01:05:38Z

PR_Github #66068 [ run ] completed with state SUCCESS. Commit: 3392cd6
/LLM/main/L0_MergeRequest_PR pipeline #53748 (Partly Tested) completed with status: 'SUCCESS'

CI Report

Link to invocation

brnguyen2 · 2026-08-14T01:21:37Z

/bot run

tensorrt-cicd · 2026-08-14T01:27:33Z

PR_Github #66107 [ run ] triggered by Bot. Commit: 3392cd6 Link to invocation

tensorrt-cicd · 2026-08-14T04:41:24Z

PR_Github #66107 [ run ] completed with state FAILURE. Commit: 3392cd6
/LLM/main/L0_MergeRequest_PR pipeline #53784 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

brnguyen2 · 2026-08-14T04:46:56Z

/bot run

tensorrt-cicd · 2026-08-14T04:55:01Z

PR_Github #66174 [ run ] triggered by Bot. Commit: 3392cd6 Link to invocation

tensorrt-cicd · 2026-08-14T07:50:55Z

PR_Github #66174 [ run ] completed with state FAILURE. Commit: 3392cd6
/LLM/main/L0_MergeRequest_PR pipeline #53847 completed with status: 'UNSTABLE'

CI Report

⚠️ Multi-GPU Label Required:
Multi-GPU tests require the ci: full pre-merge approved label on this PR. Ask a member of NVIDIA/trt-llm-ci-approvers to add the label, then re-trigger CI with the same bot command (no rebase needed).

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation

brnguyen2 · 2026-08-14T07:53:25Z

/bot run

brnguyen2 · 2026-08-14T09:56:55Z

/bot run

tensorrt-cicd · 2026-08-14T10:03:10Z

PR_Github #66250 [ run ] triggered by Bot. Commit: 3392cd6 Link to invocation

brnguyen2 · 2026-08-14T13:25:52Z

/bot skip --comment "Only failure is TestLlama3_1_8BInstruct::test_tp_pp_symmetric[MMLU-tp2pp1] on DGX_H100-4_GPUs-PyTorch-Others-2 — a disaggregated-serving timeout (server hang), not an accuracy failure. This PR only touches trtllm-gen fused MoE SiTu activation-slot handling; Llama-3.1-8B is a dense model that never runs that path, and the utils.py change is confined to the trtllm-gen-local ActType_TrtllmGen enum (shared ActivationType untouched). Known main-side flake: 1.2-2.4% over 110+ distinct MRs, hitting moraxu/Tabrizian/lori-ren on the same stage. All other stages passing."

tensorrt-cicd · 2026-08-14T13:31:35Z

PR_Github #66283 [ skip ] triggered by Bot. Commit: 3392cd6 Link to invocation

tensorrt-cicd · 2026-08-14T13:33:59Z

PR_Github #66250 [ run ] completed with state ABORTED. Commit: 3392cd6

Link to invocation

tensorrt-cicd · 2026-08-14T13:39:46Z

PR_Github #66283 [ skip ] completed with state SUCCESS. Commit: 3392cd6
Skipping testing for commit 3392cd6

Link to invocation

github-actions Bot assigned brnguyen2 Aug 7, 2026

brnguyen2 marked this pull request as ready for review August 7, 2026 14:08

brnguyen2 requested review from a team as code owners August 7, 2026 14:08

brnguyen2 requested review from Barry-Delaney, allisonlim-nv, lori-ren and xxi-nv August 7, 2026 14:08

brnguyen2 force-pushed the k3/15177-situ-activation branch from e4e12fa to be08d33 Compare August 7, 2026 18:20

brnguyen2 requested a review from a team as a code owner August 9, 2026 16:46

brnguyen2 force-pushed the k3/15177-situ-activation branch from 36c8df0 to c45b584 Compare August 13, 2026 18:04

test waives: skip VANILLA BERT ptp_quickstart e2e (pre-existing timeo…

3b66f72

…ut on RTXPro6000D-PyTorch-1, tracking bug pending) Signed-off-by: Brian Nguyen <brnguyen@nvidia.com>

brnguyen2 mentioned this pull request Aug 13, 2026

[None][test] Waive 3 DGX_B200 main-side CI flakes (kv-cache eviction, qwen3 fp8 lora, gemma4 e2e) #17663

Merged

1 task

coderabbitai Bot mentioned this pull request Aug 14, 2026

[TRTLLM-15284][feat] add Kimi K3 SiTU MegaMoE support #17624

Merged

Barry-Delaney approved these changes Aug 14, 2026

View reviewed changes

brnguyen2 added the ci: full pre-merge approved label Aug 14, 2026

brnguyen2 enabled auto-merge (squash) August 14, 2026 13:26

brnguyen2 merged commit 3ef6f48 into NVIDIA:main Aug 14, 2026
14 checks passed

Uh oh!

Conversation

brnguyen2 commented Aug 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Included

Validation

Dev Engineer Review

QA Engineer Review

Note: temporary CI-flake waives

Uh oh!

brnguyen2 commented Aug 7, 2026

Uh oh!

coderabbitai Bot commented Aug 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Reviews paused

Walkthrough

Changes

Uh oh!

tensorrt-cicd commented Aug 7, 2026

Uh oh!

tensorrt-cicd commented Aug 7, 2026

Uh oh!

brnguyen2 commented Aug 7, 2026

Uh oh!

tensorrt-cicd commented Aug 7, 2026

Uh oh!

tensorrt-cicd commented Aug 7, 2026

Uh oh!

brnguyen2 commented Aug 7, 2026

Uh oh!

tensorrt-cicd commented Aug 7, 2026

Uh oh!

tensorrt-cicd commented Aug 7, 2026

Uh oh!

brnguyen2 commented Aug 7, 2026

Uh oh!

tensorrt-cicd commented Aug 7, 2026

Uh oh!

tensorrt-cicd commented Aug 7, 2026

Uh oh!

brnguyen2 commented Aug 7, 2026

Uh oh!

tensorrt-cicd commented Aug 7, 2026

Uh oh!

tensorrt-cicd commented Aug 8, 2026

Uh oh!

brnguyen2 commented Aug 8, 2026

Uh oh!

tensorrt-cicd commented Aug 8, 2026

Uh oh!

tensorrt-cicd commented Aug 8, 2026

Uh oh!

brnguyen2 commented Aug 9, 2026

Uh oh!

tensorrt-cicd commented Aug 13, 2026

Uh oh!

brnguyen2 commented Aug 13, 2026

Uh oh!

tensorrt-cicd commented Aug 13, 2026

Uh oh!

tensorrt-cicd commented Aug 13, 2026

Uh oh!

brnguyen2 commented Aug 13, 2026

Uh oh!

tensorrt-cicd commented Aug 13, 2026

Uh oh!

tensorrt-cicd commented Aug 13, 2026

Uh oh!

tensorrt-cicd commented Aug 14, 2026

Uh oh!

brnguyen2 commented Aug 14, 2026

Uh oh!

tensorrt-cicd commented Aug 14, 2026

Uh oh!

tensorrt-cicd commented Aug 14, 2026

Uh oh!

brnguyen2 commented Aug 14, 2026

Uh oh!

tensorrt-cicd commented Aug 14, 2026

brnguyen2 commented Aug 7, 2026 •

edited

Loading

coderabbitai Bot commented Aug 7, 2026 •

edited

Loading