Skip to content

Fixes #158 : avoid duplicate add Qwen3.5 shared expert output - #159

Open
johnny-zjh wants to merge 1 commit into
NVIDIA:mainfrom
johnny-zjh:qwen3_5_moe_shared_expert_fix
Open

Fixes #158 : avoid duplicate add Qwen3.5 shared expert output#159
johnny-zjh wants to merge 1 commit into
NVIDIA:mainfrom
johnny-zjh:qwen3_5_moe_shared_expert_fix

Conversation

@johnny-zjh

Copy link
Copy Markdown

What does this PR do?

Type of change: Bug fix

Overview: Fixes duplicate shared-expert accumulation in the Qwen3.5 MoE forward path. Qwen3SparseMoeBlock.forward() already combines the routed-expert output with the gated shared-expert output. Qwen3_5SparseMoeBlock.forward() previously called the base implementation and then added the same shared-expert output again, casue erros;

Usage

No user-facing API or workflow changes are required. Export Qwen3.5 MoE models using the existing documented workflow.

🚀 Pull Request Checklist

Thank you for contributing to TensorRT Edge-LLM! Before we review your pull request, please make sure the following items are complete.
Please also refer to Contributor guidelines for general guidelines.

✅ Pre-commit Checks

  • ✅ I have installed pre-commit by running pip install pre-commit.
  • ✅ I have installed the hooks with pre-commit install.
  • ✅ I have run the hooks manually with pre-commit run --all-files and fixed any reported issues.

🧪 Tests

  • Tests have been added or updated as needed.
  • All tests are passing.

📄 Documentation

  • Documentation updates are not required because this change only corrects internal model execution behavior.

⚙️ Compatibility

  • ✅ The change is backward compatible

@johnny-zjh
johnny-zjh requested a review from a team August 3, 2026 13:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant