@@ -240,29 +240,44 @@ cd .github/run-eval
240240MODEL_IDS=" your-model-id" GITHUB_OUTPUT=/tmp/output.txt python resolve_model_config.py
241241```
242242
243- ## Step 6: Run Integration Tests (Required Before PR)
243+ ## Step 6: Create Draft PR
244244
245- ** Mandatory ** : Integration tests must pass before creating PR.
245+ Push your branch and create a draft PR. Note the PR number returned - you'll need it for the integration tests .
246246
247- ### Via GitHub Actions
247+ ## Step 7: Run Integration Tests
248248
249- 1 . Push branch: ` git push origin your-branch-name `
250- 2 . Navigate to: https://github.com/OpenHands/software-agent-sdk/actions/workflows/integration-runner.yml
251- 3 . Click "Run workflow"
252- 4 . Configure:
253- - ** Branch** : Select your branch
254- - ** model_ids** : ` your-model-id `
255- - ** Reason** : "Testing model-id"
256- 5 . Wait for completion
257- 6 . ** Save run URL** - required for PR description
249+ Trigger integration tests on your PR branch:
250+
251+ ``` bash
252+ gh workflow run integration-runner.yml \
253+ -f model_ids=your-model-id \
254+ -f reason=" Testing new model from PR #<pr-number>" \
255+ -f issue_number=< pr-number> \
256+ --ref your-branch-name
257+ ```
258+
259+ Results will be posted back to the PR as a comment.
258260
259261### Expected Results
260262
261263- Success rate: 100% (or 87.5% if vision test skipped)
262264- Duration: 5-10 minutes per model
263265- Tests: 8 total (basic commands, file ops, code editing, reasoning, errors, tools, context, vision)
264266
265- ## Step 7: Create PR
267+ ## Step 8: Fix Issues and Rerun (if needed)
268+
269+ If tests fail, see [ Common Issues] ( #common-issues ) below. After fixing:
270+
271+ 1 . Push the fix: ` git add . && git commit && git push `
272+ 2 . Rerun integration tests with the same command from Step 7 (using the same PR number)
273+
274+ ## Step 9: Mark PR Ready
275+
276+ When tests pass, mark the PR as ready for review:
277+
278+ ``` bash
279+ gh pr ready < pr-number>
280+ ```
266281
267282### Required in PR Description
268283
@@ -379,3 +394,4 @@ Fixes #[issue-number]
379394- Recent model additions: #2102 , #2153 , #2207 , #2233 , #2269
380395- Common issues: #2147 (hangs), #2137 (parameters), #2110 (vision), #2233 (variants), #2193 (preflight)
381396- Integration test workflow: ` .github/workflows/integration-runner.yml `
397+ - Integration tests can be triggered via: ` gh workflow run integration-runner.yml --ref <branch> `
0 commit comments