[TRTLLM-15103][test] clean retired phi tests by xinhe-nv · Pull Request #17600 · NVIDIA/TensorRT-LLM

xinhe-nv · 2026-08-13T04:04:04Z

…][test] Prune Phi-3/Phi-4 functional and unit tests

Remove out-of-scope Phi-3, Phi-4, Phi-4-mini, Phi-4-multimodal, and Phi4-Reasoning-Plus coverage per Test Scope Review 2026.7.

Dev Engineer Review

Removed out-of-scope Phi-3, Phi-4, Phi-4-mini, Phi-4-multimodal, and Phi-4-Reasoning-Plus tests and supporting configuration.
Removed related accuracy references, model paths, fixtures, LoRA helpers, performance entries, and test-list entries.
Added Qwen3 Spark performance cases.
Generalized one model-specific test description and updated copyright years.
Configuration and test-list changes match the stated scope.
Review should confirm that no references remain to removed fixtures, model paths, or test functions.
CI validation needs follow-up. Runs #65782, #65929, and #66231 had pipeline failures. Run #66294 has no reported result.

QA Engineer Review

Removed test functions and classes from test_cli_flow.py, test_llm_api_pytorch.py, test_llm_api_pytorch_encode.py, and test_llm_api_pytorch_multimodal.py.
Deleted test_phi.py, test_modeling_phi3.py, and test_modeling_phi4mm.py.
Removed Phi-3 LoRA tests, Phi-3 auto-deploy smoke tests, Phi-4 multimodal end-to-end tests, and related helpers.
Removed corresponding entries from A100, A30, H100, RTX6K, Spark core, Spark function, and Spark performance test lists.
The affected integration tests have corresponding test-db/ or qa/ list updates where applicable. Deleted unit, example, and helper tests do not require test-list entries.
Verdict: needs follow-up because CI pipelines failed and no result is available for the latest requested run.

Description

Test Coverage

PR Checklist

Please review the following before submitting your PR:

PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.
PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.
Test cases are provided for new code paths (see test instructions)
If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.
Any new dependencies have been scanned for license and vulnerabilities
CODEOWNERS updated if ownership changes
Documentation updated as needed
Update tava architecture diagram if there is a significant design change in PR.
The reviewers assigned automatically/manually are appropriate for the PR.
Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

…][test] Prune Phi-3/Phi-4 functional and unit tests Remove out-of-scope Phi-3, Phi-4, Phi-4-mini, Phi-4-multimodal, and Phi4-Reasoning-Plus coverage per Test Scope Review 2026.7. Signed-off-by: Xin He (SW-GPU) <200704525+xinhe-nv@users.noreply.github.com>

xinhe-nv · 2026-08-13T04:04:45Z

/bot run

coderabbitai · 2026-08-13T04:05:25Z

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

@coderabbitai resume to resume automatic reviews.
@coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

▶️ Resume reviews
🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: f58c884c-ebac-4cda-8b0f-7d02d69d443c

📥 Commits

Reviewing files that changed from the base of the PR and between 69f85d4 and 1a5217d.

📒 Files selected for processing (1)

tests/integration/test_lists/waives.txt

💤 Files with no reviewable changes (1)

tests/integration/test_lists/waives.txt

Walkthrough

The changes remove Phi-3 and Phi-4 accuracy, performance, end-to-end, fixture, LoRA, and unit-test coverage. They also add Qwen3 32B performance cases and update related metadata.

Changes

Phi coverage retirement

Layer / File(s)	Summary
Remove Phi accuracy coverage `tests/integration/defs/accuracy/...`, `tests/integration/test_lists/...`	Removes Phi accuracy references, integration test classes, decoder parameters, and scheduled test entries.
Remove performance and end-to-end coverage `tests/integration/defs/perf/...`, `tests/integration/defs/test_e2e.py`, `tests/integration/test_lists/qa/llm_spark_perf.yml`	Removes Phi performance configurations and multimodal handling. Adds Qwen3 32B FP4 and BF16 streaming cases.
Remove Phi fixtures and model mappings `tests/unittest/auto_deploy/...`	Removes the Phi-3 small-model configuration and its cached-attention smoke tests.
Remove Phi LoRA unit coverage `tests/unittest/llmapi/...`, `tests/unittest/inputs/test_chat_template_dispatch.py`, `tests/integration/test_lists/test-db/l0_h100.yml`	Removes Phi LoRA helpers and multi-GPU comparison coverage. Removes the Phi-4 LoRA example entry and generalizes a multimodal test description.

Estimated code review effort: 2 (Simple) | ~10 minutes

Merge Risk: ⚪ Minimal · up to 1a521

This change removes retired Phi test coverage without any identified merge-blocking risk; it is merge-ready after normal checks and review.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly identifies the cleanup of retired Phi tests and follows the required ticket and type format.
Description check	✅ Passed	The description explains the scope and rationale, includes the required template and checklist, but does not specify test coverage.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

_{Comment @coderabbitai help to get the list of available commands.}

tensorrt-cicd · 2026-08-13T04:11:15Z

PR_Github #65782 [ run ] triggered by Bot. Commit: cd24141 Link to invocation

tensorrt-cicd · 2026-08-13T06:46:57Z

PR_Github #65782 [ run ] completed with state FAILURE. Commit: cd24141
/LLM/main/L0_MergeRequest_PR pipeline #53491 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

xinhe-nv · 2026-08-13T13:55:03Z

/bot run

tensorrt-cicd · 2026-08-13T14:01:41Z

PR_Github #65929 [ run ] triggered by Bot. Commit: 32c96ed Link to invocation

tensorrt-cicd · 2026-08-13T16:47:40Z

PR_Github #65929 [ run ] completed with state SUCCESS. Commit: 32c96ed
/LLM/main/L0_MergeRequest_PR pipeline #53621 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

xinhe-nv · 2026-08-14T08:47:12Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-08-14T08:52:58Z

PR_Github #66231 [ run ] triggered by Bot. Commit: 32c96ed Link to invocation

tensorrt-cicd · 2026-08-14T11:55:14Z

PR_Github #66231 [ run ] completed with state FAILURE. Commit: 32c96ed
/LLM/main/L0_MergeRequest_PR pipeline #53901 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

Signed-off-by: xinhe-nv <200704525+xinhe-nv@users.noreply.github.com>

xinhe-nv · 2026-08-14T14:36:39Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-08-14T14:42:20Z

PR_Github #66294 [ run ] triggered by Bot. Commit: 69f85d4 Link to invocation

The main merge reintroduced a waive for the removed test_phi3_lora_fused_modules_output_on_tp2_identical_to_tp1 entry. Signed-off-by: Xin He (SW-GPU) <200704525+xinhe-nv@users.noreply.github.com>

tensorrt-cicd · 2026-08-14T21:03:48Z

PR_Github #66294 [ run ] completed with state SUCCESS. Commit: 69f85d4
/LLM/main/L0_MergeRequest_PR pipeline #53951 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

xinhe-nv · 2026-08-15T02:40:55Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-08-15T02:47:34Z

PR_Github #66420 [ run ] triggered by Bot. Commit: 1a5217d Link to invocation

tensorrt-cicd · 2026-08-15T07:28:02Z

PR_Github #66420 [ run ] completed with state FAILURE. Commit: 1a5217d
/LLM/main/L0_MergeRequest_PR pipeline #54063 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

xinhe-nv · 2026-08-15T15:08:17Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-08-15T15:14:12Z

PR_Github #66480 [ run ] triggered by Bot. Commit: 1a5217d Link to invocation

tensorrt-cicd · 2026-08-15T17:45:22Z

PR_Github #66480 [ run ] completed with state SUCCESS. Commit: 1a5217d
/LLM/main/L0_MergeRequest_PR pipeline #54123 completed with status: 'UNSTABLE'

CI Report

⚠️ Multi-GPU Label Required:
Multi-GPU tests require the ci: full pre-merge approved label on this PR. Ask a member of NVIDIA/trt-llm-ci-approvers to add the label, then re-trigger CI with the same bot command (no rebase needed).

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation

xinhe-nv requested review from a team as code owners August 13, 2026 04:04

xinhe-nv requested review from QiJune, StanleySun639, YihuiLu512, bmarimuthu-nv, jieli-matrix, mzweilz, niukuo and schetlur-nv August 13, 2026 04:04

github-actions Bot assigned xinhe-nv Aug 13, 2026

Merge branch 'main' into trtllm-15103

cd24141

xinhe-nv enabled auto-merge (squash) August 13, 2026 04:04

BowenFu approved these changes Aug 13, 2026

View reviewed changes

coderabbitai Bot mentioned this pull request Aug 13, 2026

[TRTLLM-15099][test] Prune Mistral functional and unit tests #17598

Open

1 task

xinhe-nv added 2 commits August 13, 2026 15:07

Merge branch 'main' into trtllm-15103

f49f862

Merge branch 'main' into trtllm-15103

32c96ed

coderabbitai Bot mentioned this pull request Aug 13, 2026

[TRTLLMINF-133][infra] Move check testlist stage earlier in CI #17513

Open

1 task

allisonlim-nv approved these changes Aug 13, 2026

View reviewed changes

StanleySun639 approved these changes Aug 14, 2026

View reviewed changes

Merge branch 'main' into trtllm-15103

69f85d4

Signed-off-by: xinhe-nv <200704525+xinhe-nv@users.noreply.github.com>

[TRTLLM-15103][test] Drop stale Phi-3 LoRA waive after main merge

1a5217d

The main merge reintroduced a waive for the removed test_phi3_lora_fused_modules_output_on_tp2_identical_to_tp1 entry. Signed-off-by: Xin He (SW-GPU) <200704525+xinhe-nv@users.noreply.github.com>

Uh oh!

Conversation

xinhe-nv commented Aug 13, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Dev Engineer Review

QA Engineer Review

Description

Test Coverage

PR Checklist

GitHub Bot Help

Uh oh!

xinhe-nv commented Aug 13, 2026

Uh oh!

coderabbitai Bot commented Aug 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Reviews paused

Walkthrough

Changes

Uh oh!

tensorrt-cicd commented Aug 13, 2026

Uh oh!

tensorrt-cicd commented Aug 13, 2026

Uh oh!

xinhe-nv commented Aug 13, 2026

Uh oh!

tensorrt-cicd commented Aug 13, 2026

Uh oh!

tensorrt-cicd commented Aug 13, 2026

Uh oh!

xinhe-nv commented Aug 14, 2026

Uh oh!

tensorrt-cicd commented Aug 14, 2026

Uh oh!

tensorrt-cicd commented Aug 14, 2026

Uh oh!

xinhe-nv commented Aug 14, 2026

Uh oh!

tensorrt-cicd commented Aug 14, 2026

Uh oh!

tensorrt-cicd commented Aug 14, 2026

Uh oh!

xinhe-nv commented Aug 15, 2026

Uh oh!

tensorrt-cicd commented Aug 15, 2026

Uh oh!

tensorrt-cicd commented Aug 15, 2026

Uh oh!

xinhe-nv commented Aug 15, 2026

Uh oh!

tensorrt-cicd commented Aug 15, 2026

Uh oh!

tensorrt-cicd commented Aug 15, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

xinhe-nv commented Aug 13, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Aug 13, 2026 •

edited

Loading