[https://nvbugs/6428063][fix] Include use_host_stop_criteria in the PP ring-broadcast payload (sender uses… - #16136
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: 📒 Files selected for processing (2)
💤 Files with no reviewable changes (1)
🚧 Files skipped from review as they are similar to previous changes (1)
WalkthroughThis PR adds ChangesRing Broadcast Protocol Update
Test Waiver Cleanup
Estimated code review effort: 2 (Simple) | ~10 minutes Merge Risk: ⚪ Minimal · up to The change synchronizes sampler stop-criteria behavior across pipeline-parallel ranks and removes the related waiver; no actionable merge-blocking risk remains beyond normal checks and review. Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
41705fc to
e2bfdb4
Compare
PR NVIDIA#15920 introduced a fast path in TorchSampler that skips writing finish_reasons to host when all requests are greedy, single-beam, and have no stop words. In pipeline-parallel mode with disable_overlap_scheduler=False, only the last PP rank runs sample_async and knows the fast path was taken; non-last PP ranks construct SampleState in _forward_step_inter_pp with the default use_host_stop_criteria=False. The ring-broadcast payload sent only sample_state.host, so non-last ranks ended up with finish_reasons=None but use_host_stop_criteria=False, falling through to process_draft_tokens -> finish_if_reason which indexed an empty list and raised IndexError. Include use_host_stop_criteria in the PP ring-broadcast payload so all ranks agree on which update_requests branch to take. Guard with getattr/hasattr so SampleStateTRTLLM (which lacks this field) is unaffected. Remove the waiver for the affected test. Signed-off-by: trtllm-agent <296075020+trtllm-agent@users.noreply.github.com>
e2bfdb4 to
5dcd222
Compare
|
Note GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer. |
Summary
Test plan
Links
Dev Engineer Review
SampleStateTorch.use_host_stop_criteriaacross pipeline-parallel ranks.getattr(..., False)for send-side compatibility and anhasattrguard on receive.QA Engineer Review
tests/integration/test_lists/waives.txt.accuracy/test_llm_api_pytorch.py::TestDeepSeekV3Lite::test_nvfp4_4gpus, tracked bynvbugs/6428063.