test: add support matrix per-PR test subset

[Harrilee]

Add a per-PR test for support matrix.

Assumptions

1. Only the latest inference engine version is tested
2. Only the B200 and H100 is tested
3. If support_matrix.csv does not support, will skip it.

Models Under Test

1. "nvidia/DeepSeek-V3.1-NVFP4",
2. "meta-llama/Meta-Llama-3.1-8B",
3. "MiniMaxAI/MiniMax-M2.5",
4. "nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16",
5. "Qwen/Qwen3-235B-A22B",
6. "openai/gpt-oss-20b",

Perf

1. This modification will add additional 5 minutes to the originally 8-minute test.

[coderabbitai]

Walkthrough

A new pytest marker support_matrix is declared in pytest.ini for PR-level smoke tests. A new comprehensive end-to-end test file validates model/system/backend/version combinations against the expected support matrix, using parametrized tests and helper functions to determine supported execution modes.

Changes

Cohort / File(s)	Summary
Configuration pytest.ini	Added new pytest marker declaration support_matrix for PR-level support matrix smoke tests.
New E2E Test Suite tests/e2e/support_matrix/test_pr_support_matrix.py	Added comprehensive parametrized test file with test_pr_support_matrix() function that validates model/system/backend/version combinations. Includes helper functions _latest_version() for caching latest database versions and _build_param_grid() for constructing test parameter combinations across fixed models, supported systems, available backends, and versions.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Poem

🐰 A matrix of tests, now hopping with care,
Backends and systems dance in the air,
Models aligned with support so true,
PR smoke tests validate through and through!
Combinations checked, regressions caught fast—
Quality assured, built to last! ✨

🚥 Pre-merge checks | ✅ 1 | ❌ 2

❌ Failed checks (2 warnings)

Check name	Status	Explanation	Resolution
Description check	⚠️ Warning	No pull request description was provided by the author; all required template sections (Overview, Details, Where should the reviewer start, Related Issues) are missing.	Add a comprehensive description following the template: explain the purpose of the support matrix PR test, detail the changes in pytest.ini and the new test file, indicate which files should be reviewed closely, and link any related issues.
Docstring Coverage	⚠️ Warning	Docstring coverage is 66.67% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (1 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title accurately reflects the main change: adding a per-PR test subset for the support matrix, which is directly evidenced by the new test file and pytest marker.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

📝 Coding Plan

• Generate coding plan for human review comments

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

Tip

You can generate walkthrough in a markdown collapsible section to save space.

Enable the reviews.collapse_walkthrough setting to generate walkthrough in a markdown collapsible section.

[coderabbitai]

🧹 Nitpick comments (1)

tests/e2e/support_matrix/test_pr_support_matrix.py (1)

56-58: Consider logging when a (system, backend) pair has no database.

When _latest_version returns None, the combination is silently excluded from the test matrix. This is reasonable since not all backends may have databases for all systems, but adding a debug-level log could help developers understand test coverage gaps.

🔧 Optional: Add logging for skipped combinations

+import logging
+
+logger = logging.getLogger(__name__)
+
 def _build_param_grid() -> list[pytest.param]:
     """Build a flat list of pytest params for every valid (model, system, backend, version) combo."""
     params: list[pytest.param] = []
     for model in PR_MODELS:
         short_model = model.rsplit("/", 1)[-1]
         for system in PR_SYSTEMS:
             for backend in PR_BACKENDS:
                 version = _latest_version(system, backend)
                 if version is None:
+                    logger.debug("No database version for %s/%s, skipping", system, backend)
                     continue

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@tests/e2e/support_matrix/test_pr_support_matrix.py` around lines 56 - 58,
When iterating combinations, add a debug-level log when _latest_version(system,
backend) returns None so skipped (system, backend) pairs are visible; inside the
block where you currently do "if version is None: continue" call the test logger
(e.g. logger.debug or logging.getLogger(...).debug) with a concise message
including system and backend identifiers and that no database/version was found,
then continue. Ensure you reference the same variables (_latest_version,
version, system, backend) so the log is added next to the existing check without
changing control flow.

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@tests/e2e/support_matrix/test_pr_support_matrix.py`:
- Around line 56-58: When iterating combinations, add a debug-level log when
_latest_version(system, backend) returns None so skipped (system, backend) pairs
are visible; inside the block where you currently do "if version is None:
continue" call the test logger (e.g. logger.debug or
logging.getLogger(...).debug) with a concise message including system and
backend identifiers and that no database/version was found, then continue.
Ensure you reference the same variables (_latest_version, version, system,
backend) so the log is added next to the existing check without changing control
flow.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: e24489af-5b1b-4d78-99e9-1308a228268a

📥 Commits

Reviewing files that changed from the base of the PR and between 0b9726d and 3770e5c.

📒 Files selected for processing (2)

• pytest.ini
• tests/e2e/support_matrix/test_pr_support_matrix.py