Skip to content

Conversation

@xinhe-nv
Copy link
Collaborator

@xinhe-nv xinhe-nv commented Oct 12, 2025

waive failed cases.

Summary by CodeRabbit

  • Tests
    • Updated integration test selection to replace an older GPU/runtime scenario with a newer FP8 configuration using compilation enabled.
    • Aligns validation with current model/runtime settings and maintains test coverage for performance-sensitive paths.
    • No impact on product functionality or user experience; changes affect test execution only.

@xinhe-nv xinhe-nv force-pushed the user/qa/post_update_waive_20251012_LLM_FUNCTION_TEST_1484 branch from 74a0d13 to 5768f7e Compare October 13, 2025 02:37
@xinhe-nv xinhe-nv marked this pull request as ready for review October 13, 2025 02:37
@xinhe-nv xinhe-nv enabled auto-merge (squash) October 13, 2025 02:37
@xinhe-nv
Copy link
Collaborator Author

/bot run --skip-test

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Oct 13, 2025

📝 Walkthrough

Walkthrough

Updated integration test waivers by removing one specific test case and adding another in tests/integration/test_lists/waives.txt. No other files or control flow were modified.

Changes

Cohort / File(s) Summary of Changes
Integration test waivers
tests/integration/test_lists/waives.txt
Replaced accuracy/test_llm_api_pytorch.py::TestGPTOSS::test_w4_1gpu[True-True-trtllm-auto] with accuracy/test_llm_api_pytorch.py::TestLlama4MaverickInstruct::test_fp8_eagle3[tp8-torch_compile=True].

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~2 minutes

Possibly related PRs

Suggested reviewers

  • crazydemo
  • LarryXFly

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Description Check ⚠️ Warning The pull request description does not follow the repository’s required template and lacks all mandated section headings such as ## Description, ## Test Coverage, and the PR Checklist. It only contains a brief phrase without explaining the rationale for the changes, listing relevant tests, or confirming adherence to guidelines, so it fails to provide the structured information needed for review. Please update the PR description to follow the repository template by including the ## Description section that explains the problem and solution, a ## Test Coverage section listing relevant tests, and a PR Checklist confirming guideline compliance. Populate each section with meaningful details so reviewers can understand the change, its rationale, and the tests that safeguard it.
✅ Passed checks (2 passed)
Check name Status Explanation
Title Check ✅ Passed The title clearly and concisely describes the primary change—adding failed test cases into the waives.txt file—and follows the repository’s naming convention with the “[None][chore]” prefix, making it easy for teammates to understand the pull request’s intent at a glance.
Docstring Coverage ✅ Passed No functions found in the changes. Docstring coverage check skipped.
✨ Finishing touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

📜 Recent review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 989c25f and 5768f7e.

📒 Files selected for processing (1)
  • tests/integration/test_lists/waives.txt (1 hunks)
🧰 Additional context used
🧠 Learnings (4)
📓 Common learnings
Learnt from: tongyuantongyu
PR: NVIDIA/TensorRT-LLM#7781
File: tests/integration/test_lists/waives.txt:313-313
Timestamp: 2025-09-17T02:48:52.732Z
Learning: In TensorRT-LLM, `tests/integration/test_lists/waives.txt` is specifically for waiving/skipping tests, while other test list files like those in `test-db/` and `qa/` directories are for different test execution contexts (pre-merge, post-merge, QA tests). The same test appearing in both waives.txt and execution list files is intentional - the test is part of test suites but will be skipped due to the waiver.
📚 Learning: 2025-09-17T02:48:52.732Z
Learnt from: tongyuantongyu
PR: NVIDIA/TensorRT-LLM#7781
File: tests/integration/test_lists/waives.txt:313-313
Timestamp: 2025-09-17T02:48:52.732Z
Learning: In TensorRT-LLM, `tests/integration/test_lists/waives.txt` is specifically for waiving/skipping tests, while other test list files like those in `test-db/` and `qa/` directories are for different test execution contexts (pre-merge, post-merge, QA tests). The same test appearing in both waives.txt and execution list files is intentional - the test is part of test suites but will be skipped due to the waiver.

Applied to files:

  • tests/integration/test_lists/waives.txt
📚 Learning: 2025-09-09T09:40:45.658Z
Learnt from: fredricz-20070104
PR: NVIDIA/TensorRT-LLM#7645
File: tests/integration/test_lists/qa/llm_function_core.txt:648-648
Timestamp: 2025-09-09T09:40:45.658Z
Learning: In TensorRT-LLM test lists, it's common and intentional for the same test to appear in multiple test list files when they serve different purposes (e.g., llm_function_core.txt for comprehensive core functionality testing and llm_function_core_sanity.txt for quick sanity checks). This duplication allows tests to be run in different testing contexts.

Applied to files:

  • tests/integration/test_lists/waives.txt
📚 Learning: 2025-08-29T14:07:45.863Z
Learnt from: EmmaQiaoCh
PR: NVIDIA/TensorRT-LLM#7370
File: tests/unittest/trt/model_api/test_model_quantization.py:24-27
Timestamp: 2025-08-29T14:07:45.863Z
Learning: In TensorRT-LLM's CI infrastructure, pytest skip markers (pytest.mark.skip) are properly honored even when test files have __main__ blocks that call test functions directly. The testing system correctly skips tests without requiring modifications to the __main__ block execution pattern.

Applied to files:

  • tests/integration/test_lists/waives.txt
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: Pre-commit Check
🔇 Additional comments (1)
tests/integration/test_lists/waives.txt (1)

363-363: Addition looks good

Waiving the torch_compile=True variant alongside the existing False case keeps the list consistent with bug 5546510 coverage. LGTM.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Oct 13, 2025

📝 Walkthrough

Walkthrough

Updated the integration test skip list by replacing one skipped test entry with another, maintaining the SKIP status and context in tests/integration/test_lists/waives.txt.

Changes

Cohort / File(s) Summary
Test skip list update
tests/integration/test_lists/waives.txt
Replaced skipped entry accuracy/test_llm_api_pytorch.py::TestGPTOSS::test_w4_1gpu[True-True-trtllm-auto] SKIP with accuracy/test_llm_api_pytorch.py::TestLlama4MaverickInstruct::test_fp8_eagle3[tp8-torch_compile=True] SKIP.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~2 minutes

Possibly related PRs

Suggested reviewers

  • crazydemo
  • LarryXFly

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Description Check ⚠️ Warning The PR description consists of a single, generic sentence and does not follow the required template, omitting the detailed Description, Test Coverage, and PR Checklist sections specified in the repository guidance. Please revise the PR description to use the provided template by adding a clear Description of the issue and solution, listing the relevant Test Coverage entries, and completing the PR Checklist section to ensure all required information is included.
✅ Passed checks (2 passed)
Check name Status Explanation
Title Check ✅ Passed The title succinctly describes the main change of adding failed test cases to the waives.txt file and follows the required “[None][chore] Summary” format, clearly indicating the type and scope of the update.
Docstring Coverage ✅ Passed No functions found in the changes. Docstring coverage check skipped.
✨ Finishing touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

📜 Recent review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 989c25f and 5768f7e.

📒 Files selected for processing (1)
  • tests/integration/test_lists/waives.txt (1 hunks)
🧰 Additional context used
🧠 Learnings (3)
📓 Common learnings
Learnt from: tongyuantongyu
PR: NVIDIA/TensorRT-LLM#7781
File: tests/integration/test_lists/waives.txt:313-313
Timestamp: 2025-09-17T02:48:52.732Z
Learning: In TensorRT-LLM, `tests/integration/test_lists/waives.txt` is specifically for waiving/skipping tests, while other test list files like those in `test-db/` and `qa/` directories are for different test execution contexts (pre-merge, post-merge, QA tests). The same test appearing in both waives.txt and execution list files is intentional - the test is part of test suites but will be skipped due to the waiver.
📚 Learning: 2025-09-17T02:48:52.732Z
Learnt from: tongyuantongyu
PR: NVIDIA/TensorRT-LLM#7781
File: tests/integration/test_lists/waives.txt:313-313
Timestamp: 2025-09-17T02:48:52.732Z
Learning: In TensorRT-LLM, `tests/integration/test_lists/waives.txt` is specifically for waiving/skipping tests, while other test list files like those in `test-db/` and `qa/` directories are for different test execution contexts (pre-merge, post-merge, QA tests). The same test appearing in both waives.txt and execution list files is intentional - the test is part of test suites but will be skipped due to the waiver.

Applied to files:

  • tests/integration/test_lists/waives.txt
📚 Learning: 2025-08-29T14:07:45.863Z
Learnt from: EmmaQiaoCh
PR: NVIDIA/TensorRT-LLM#7370
File: tests/unittest/trt/model_api/test_model_quantization.py:24-27
Timestamp: 2025-08-29T14:07:45.863Z
Learning: In TensorRT-LLM's CI infrastructure, pytest skip markers (pytest.mark.skip) are properly honored even when test files have __main__ blocks that call test functions directly. The testing system correctly skips tests without requiring modifications to the __main__ block execution pattern.

Applied to files:

  • tests/integration/test_lists/waives.txt
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: Pre-commit Check
🔇 Additional comments (1)
tests/integration/test_lists/waives.txt (1)

363-363: Approve waiver addition for torch_compile=True variant.
No existing waiver for this exact variant; format and bug reference match existing entries.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@tensorrt-cicd
Copy link
Collaborator

PR_Github #21127 [ run ] triggered by Bot

@xinhe-nv xinhe-nv force-pushed the user/qa/post_update_waive_20251012_LLM_FUNCTION_TEST_1484 branch from e8ecd70 to 402da1b Compare October 13, 2025 04:08
@tensorrt-cicd
Copy link
Collaborator

PR_Github #21127 [ run ] completed with state SUCCESS
/LLM/main/L0_MergeRequest_PR pipeline #15960 (Partly Tested) completed with status: 'SUCCESS'
Pipeline passed with automatic retried tests. Check the rerun report for details.

Signed-off-by: xinhe-nv <[email protected]>
Signed-off-by: Xin He (SW-GPU) <[email protected]>
@xinhe-nv xinhe-nv force-pushed the user/qa/post_update_waive_20251012_LLM_FUNCTION_TEST_1484 branch from 402da1b to c3a2685 Compare October 13, 2025 06:26
@xinhe-nv
Copy link
Collaborator Author

/bot reuse-pipeline

@tensorrt-cicd
Copy link
Collaborator

PR_Github #21154 [ reuse-pipeline ] triggered by Bot

@tensorrt-cicd
Copy link
Collaborator

PR_Github #21154 [ reuse-pipeline ] completed with state SUCCESS
Reusing PR_Github #21127 (Partly Tested) for commit c3a2685

@xinhe-nv xinhe-nv merged commit 9fe63dd into NVIDIA:main Oct 13, 2025
5 checks passed
@xinhe-nv xinhe-nv deleted the user/qa/post_update_waive_20251012_LLM_FUNCTION_TEST_1484 branch October 13, 2025 07:29
yufeiwu-nv pushed a commit to yufeiwu-nv/TensorRT-LLM that referenced this pull request Oct 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants