fix: remove noisy qwen2 vl nightly test loss check

[terrykong]

What does this PR do ?

The loss check for this test does not follow any particular trend and just hovers. In general we shouldn't use train/loss in grpo nightlies:

https://wandb.ai/nvidia/nemo-rl/panel/952wk61kd?nw=u4eso5k9jwb

Issues

List issues that this PR closes (syntax):

Usage

• You can potentially add a usage example below

# Add a code snippet demonstrating how to use this

Before your PR is "Ready for review"

Pre checks:

• Make sure you read and followed Contributor guidelines
• Did you write any new necessary tests?
• Did you run the unit tests and functional tests locally? Visit our Testing Guide for how to run tests
• Did you add or update any necessary documentation? Visit our Document Development Guide for how to write, build and test the docs.

Additional Information

• ...

Summary by CodeRabbit

• Tests
  • Simplified automated validation by removing the loss threshold check; success is now determined solely by meeting the reward metric at the target step.
  • Aligns test success criteria with observed training outcomes for more stable test behavior.
  • No user-facing functionality changes.

[coderabbitai]

📝 Walkthrough

Walkthrough

Adjusted a VLM test suite shell script to modify metric checks passed to tests/check_metrics.py: removed the train/loss threshold at step 200, retaining only the train/reward threshold at step 200.

Changes

Cohort / File(s)	Summary
Test suite metric check tests/test_suites/vlm/vlm_grpo-qwen2.5-vl-3b-instruct-clevr-1n2g-dtensor2tp1.v1.sh	Updated uv run tests/check_metrics.py invocation to drop data["train/loss"]["200"] < 0.1 condition; kept data["train/reward"]["200"] > 0.9 only.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  participant CI as CI Runner
  participant Sh as Test Suite (.sh)
  participant CM as check_metrics.py

  CI->>Sh: Execute VLM test suite
  Sh->>CM: Invoke with metric checks
  Note over Sh,CM: Change: Only reward@step200 checked (loss@step200 removed)
  CM->>Sh: Return pass/fail based on reward condition
  Sh->>CI: Exit code reflects result

Loading

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~6 minutes

Suggested labels

CI:L0

Suggested reviewers

• yfw
• chtruong814
• guyueh1

Pre-merge checks and finishing touches

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title Check	✅ Passed	The title succinctly describes the main change—removing the noisy loss check from the Qwen2 VL nightly test—and uses clear, specific language that aligns with the actual modifications in the pull request.
Docstring Coverage	✅ Passed	No functions found in the changes. Docstring coverage check skipped.
Test Results For Major Changes	✅ Passed	The PR modifies a single nightly test script by removing a loss-threshold assertion while retaining the reward check, which is a small adjustment to test criteria rather than a major feature or performance change, and the PR description does not list test results. Because the modification is minor and does not necessitate documented testing per the custom check instructions, the absence of test results in the description is acceptable. Therefore, the custom check condition is satisfied.

✨ Finishing touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch tk/qwen2vl-unstable

---

📜 Recent review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between d0e203c and 5677113.

📒 Files selected for processing (1)

• tests/test_suites/vlm/vlm_grpo-qwen2.5-vl-3b-instruct-clevr-1n2g-dtensor2tp1.v1.sh (0 hunks)

💤 Files with no reviewable changes (1)

• tests/test_suites/vlm/vlm_grpo-qwen2.5-vl-3b-instruct-clevr-1n2g-dtensor2tp1.v1.sh

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)

• GitHub Check: semantic-pull-request / semantic-pull-request
• GitHub Check: Lint check
• GitHub Check: Post submodule check comment / Comment on PR
• GitHub Check: Post automodel integration comment / Comment on PR

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}