RHDH Add CI configuration for RHDH release-1.10 branch#79332
Conversation
WalkthroughThis PR adds a new ci-operator configuration for redhat-developer/rhdh release-1.10, defining build_root, release metadata, global resource defaults, a one-time e2e job, and a broad matrix of nightly E2E tests across OCP and multi-cloud platforms with scheduling, env injection, reporter, and workflow/post wiring. ChangesRHDH Release 1.10 CI Pipeline
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Suggested labels
Suggested reviewers
🚥 Pre-merge checks | ✅ 12✅ Passed checks (12 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
zdrapela
changed the title
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@ci-operator/config/redhat-developer/rhdh/redhat-developer-rhdh-release-1.10.yaml`:
- Around line 41-55: Several nightly jobs marked presubmit: true (for example
the job with as: e2e-ocp-helm-nightly) lack an explicit max_concurrency setting;
add a max_concurrency: <number> field to each presubmit: true nightly job block
to cap concurrent runs (choose an appropriate integer like 1 or 2 for high-cost
cloud jobs), ensuring the new key sits alongside cron/optional/presubmit fields
in each job definition so concurrency is enforced for jobs such as the blocks
containing as:, cron:, and presubmit: true entries listed in the diff.
- Around line 340-361: The job e2e-osd-gcp-operator-nightly is currently
referencing the Helm test ref redhat-developer-rhdh-osd-gcp-helm-nightly so the
operator scenario never runs; update the job's test refs under the test: block
to point to the operator test ref (replace
redhat-developer-rhdh-osd-gcp-helm-nightly with the correct operator test ref,
e.g., redhat-developer-rhdh-osd-gcp-operator-nightly) or add the operator ref
alongside Helm if both should run, making sure the job name
e2e-osd-gcp-operator-nightly maps to the operator test implementation.
- Line 52: The cron schedule for the 4.18 nightly is currently set to run daily
("cron: 0 5 * * *") which conflicts with the release-branch matrix cadence;
update the cron key to the staggered schedule used by the branch matrix (e.g. "0
5 * * MON,WED,FRI" or equivalent "0 5 * * 1,3,5") so the nightly runs only on
Monday, Wednesday and Friday to avoid resource contention.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository YAML (base), Central YAML (inherited)
Review profile: CHILL
Plan: Enterprise
Run ID: 054ee86b-f185-4dac-9dc6-4958d4af43ea
⛔ Files ignored due to path filters (2)
ci-operator/jobs/redhat-developer/rhdh/redhat-developer-rhdh-release-1.10-periodics.yamlis excluded by!ci-operator/jobs/**ci-operator/jobs/redhat-developer/rhdh/redhat-developer-rhdh-release-1.10-presubmits.yamlis excluded by!ci-operator/jobs/**
📒 Files selected for processing (1)
ci-operator/config/redhat-developer/rhdh/redhat-developer-rhdh-release-1.10.yaml
| product: ocp | ||
| timeout: 1h0m0s | ||
| version: "4.18" | ||
| cron: 0 5 * * * |
There was a problem hiding this comment.
Align the 4.18 nightly cron with the release-branch cadence.
Line 52 uses a daily schedule, but the release branch matrix here is otherwise constrained to MON,WED,FRI. This can reintroduce the resource-contention pattern the staggered plan is trying to avoid.
Suggested fix
- cron: 0 5 * * *
+ cron: 0 5 * * MON,WED,FRI📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| cron: 0 5 * * * | |
| cron: 0 5 * * 1,3,5 |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In
`@ci-operator/config/redhat-developer/rhdh/redhat-developer-rhdh-release-1.10.yaml`
at line 52, The cron schedule for the 4.18 nightly is currently set to run daily
("cron: 0 5 * * *") which conflicts with the release-branch matrix cadence;
update the cron key to the staggered schedule used by the branch matrix (e.g. "0
5 * * MON,WED,FRI" or equivalent "0 5 * * 1,3,5") so the nightly runs only on
Monday, Wednesday and Friday to avoid resource contention.
There was a problem hiding this comment.
They shouldn't run at the same time
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Check the full 1.10 config cron schedules
grep -n "cron" ci-operator/config/redhat-developer/rhdh/redhat-developer-rhdh-release-1.10.yaml
echo "---"
# Check the 1.9 config cron schedules for comparison
fd "release-1.9.yaml" ci-operator/config/redhat-developer/rhdh/ --exec grep -n "cron" {}
Length of output: 989
🏁 Script executed:
#!/bin/bash
# Get context around the 0 5 * * * job in release-1.9 to identify its cluster_claim
fd "release-1.9.yaml" ci-operator/config/redhat-developer/rhdh/ --exec sed -n '40,80p' {}
Length of output: 1014
@zdrapela, thanks for the response — but based on the actual cron values in both config files, the two jobs do appear to be scheduled at exactly the same time:
| Branch | Job | Cluster Pool | Schedule |
|---|---|---|---|
| release-1.9 | e2e-ocp-helm-nightly |
aws / us-east-2 / ocp / 4.18 |
0 5 * * * |
| release-1.10 | e2e-ocp-helm-nightly |
aws / us-east-2 / ocp / 4.18 |
0 5 * * * |
Both use the same OCP 4.18 / AWS us-east-2 cluster pool and fire at 05:00 UTC every day. They will compete for the same cluster claim simultaneously.
Since the PR description explicitly calls out OCP 4.18 large-pool jobs as spread across 03:00–05:00 UTC, it looks like the intent for 1.10 was to land at the end of that window (05:00), but release-1.9 already occupies that exact slot daily.
The simplest fix would be to either:
- Offset the 1.10 job by 30–60 minutes:
cron: 0 6 * * * - Or restrict both to
MON,WED,FRIwith a time offset, consistent with the rest of the release-branch matrix.
Would you like to re-examine the intended schedule?
| as: e2e-osd-gcp-operator-nightly | ||
| cron: 0 7 * * MON,WED,FRI | ||
| max_concurrency: 1 | ||
| optional: true | ||
| presubmit: true | ||
| reporter_config: | ||
| channel: '#rhdh-e2e-alerts-1-10' | ||
| job_states_to_report: | ||
| - error | ||
| report_template: '<!subteam^S07BMJ56R8S> <@U08UP0REWG1> :failed: `{{.Spec.Job}}` | ||
| ended with *{{.Status.State}}*, :scroll: <{{.Status.URL}}| logs>' | ||
| steps: | ||
| allow_skip_on_success: true | ||
| env: | ||
| OSD_VERSION: 4.20.17 | ||
| post: | ||
| - ref: redhat-developer-rhdh-send-data-router | ||
| - ref: redhat-developer-rhdh-send-alert | ||
| - ref: redhat-developer-rhdh-osd-gcp-claim-cluster-delete | ||
| test: | ||
| - ref: redhat-developer-rhdh-osd-gcp-helm-nightly | ||
| workflow: redhat-developer-rhdh-osd-gcp-claim-cluster |
There was a problem hiding this comment.
Operator nightly is wired to the Helm test ref.
Line 360 points e2e-osd-gcp-operator-nightly to redhat-developer-rhdh-osd-gcp-helm-nightly, so the operator scenario is not actually exercised by this job.
Suggested fix
test:
- - ref: redhat-developer-rhdh-osd-gcp-helm-nightly
+ - ref: redhat-developer-rhdh-osd-gcp-operator-nightly
workflow: redhat-developer-rhdh-osd-gcp-claim-cluster📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| as: e2e-osd-gcp-operator-nightly | |
| cron: 0 7 * * MON,WED,FRI | |
| max_concurrency: 1 | |
| optional: true | |
| presubmit: true | |
| reporter_config: | |
| channel: '#rhdh-e2e-alerts-1-10' | |
| job_states_to_report: | |
| - error | |
| report_template: '<!subteam^S07BMJ56R8S> <@U08UP0REWG1> :failed: `{{.Spec.Job}}` | |
| ended with *{{.Status.State}}*, :scroll: <{{.Status.URL}}| logs>' | |
| steps: | |
| allow_skip_on_success: true | |
| env: | |
| OSD_VERSION: 4.20.17 | |
| post: | |
| - ref: redhat-developer-rhdh-send-data-router | |
| - ref: redhat-developer-rhdh-send-alert | |
| - ref: redhat-developer-rhdh-osd-gcp-claim-cluster-delete | |
| test: | |
| - ref: redhat-developer-rhdh-osd-gcp-helm-nightly | |
| workflow: redhat-developer-rhdh-osd-gcp-claim-cluster | |
| as: e2e-osd-gcp-operator-nightly | |
| cron: 0 7 * * MON,WED,FRI | |
| max_concurrency: 1 | |
| optional: true | |
| presubmit: true | |
| reporter_config: | |
| channel: '#rhdh-e2e-alerts-1-10' | |
| job_states_to_report: | |
| - error | |
| report_template: '<!subteam^S07BMJ56R8S> <@U08UP0REWG1> :failed: `{{.Spec.Job}}` | |
| ended with *{{.Status.State}}*, :scroll: <{{.Status.URL}}| logs>' | |
| steps: | |
| allow_skip_on_success: true | |
| env: | |
| OSD_VERSION: 4.20.17 | |
| post: | |
| - ref: redhat-developer-rhdh-send-data-router | |
| - ref: redhat-developer-rhdh-send-alert | |
| - ref: redhat-developer-rhdh-osd-gcp-claim-cluster-delete | |
| test: | |
| - ref: redhat-developer-rhdh-osd-gcp-operator-nightly | |
| workflow: redhat-developer-rhdh-osd-gcp-claim-cluster |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In
`@ci-operator/config/redhat-developer/rhdh/redhat-developer-rhdh-release-1.10.yaml`
around lines 340 - 361, The job e2e-osd-gcp-operator-nightly is currently
referencing the Helm test ref redhat-developer-rhdh-osd-gcp-helm-nightly so the
operator scenario never runs; update the job's test refs under the test: block
to point to the operator test ref (replace
redhat-developer-rhdh-osd-gcp-helm-nightly with the correct operator test ref,
e.g., redhat-developer-rhdh-osd-gcp-operator-nightly) or add the operator ref
alongside Helm if both should run, making sure the job name
e2e-osd-gcp-operator-nightly maps to the operator test implementation.
There was a problem hiding this comment.
@zdrapela, thanks for confirming! Please apply the fix to point e2e-osd-gcp-operator-nightly to the correct operator test ref:
test:
- - ref: redhat-developer-rhdh-osd-gcp-helm-nightly
+ - ref: redhat-developer-rhdh-osd-gcp-operator-nightly
workflow: redhat-developer-rhdh-osd-gcp-claim-clusterWithout this change, the operator scenario will never actually run — it will silently execute the Helm test path instead.
zdrapela
force-pushed
the
rhdh-release-1.10
branch
from
3c19956 to
fc74d39
Compare
|
/pj-rehearse periodic-ci-redhat-developer-rhdh-release-1.10-e2e-ocp-helm-nightly |
|
@zdrapela: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
Add CI operator config and generated Prow jobs for the new redhat-developer/rhdh release-1.10 branch. Based on the main branch config with release-branch adaptations: - Slack alerts channel: #rhdh-e2e-alerts-1-10 - Cron schedules on MON,WED,FRI, staggered from 00:00 UTC to avoid conflicts with release-1.9 on shared resources (GKE cluster, OSD-GCP quota, small OCP cluster pools) - Cleanup jobs excluded (only run on main) - max_concurrency set on all presubmit jobs Assisted-by: OpenCode
zdrapela
force-pushed
the
rhdh-release-1.10
branch
from
fc74d39 to
ca92900
Compare
|
/pj-rehearse ack |
|
@zdrapela: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
openshift-merge-bot
Bot
added
rehearsals-ack
|
[REHEARSALNOTIFIER]
A total of 31 jobs have been affected by this change. The above listing is non-exhaustive and limited to 25 jobs. A full list of affected jobs can be found here Interacting with pj-rehearseComment: Once you are satisfied with the results of the rehearsals, comment: |
There was a problem hiding this comment.
♻️ Duplicate comments (2)
ci-operator/config/redhat-developer/rhdh/redhat-developer-rhdh-release-1.10.yaml (2)
53-53:⚠️ Potential issue | 🟠 Major | ⚡ Quick winAlign this nightly cron to the release-branch cadence.
Line 53 is daily (
0 5 * * *) while the rest of the release-branch matrix isMON,WED,FRI, which can reintroduce cross-branch contention.Suggested fix
- cron: 0 5 * * * + cron: 0 5 * * MON,WED,FRI🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@ci-operator/config/redhat-developer/rhdh/redhat-developer-rhdh-release-1.10.yaml` at line 53, Update the cron schedule for the release job by replacing the current cron value "0 5 * * *" with a release-branch cadence that runs Mondays, Wednesdays, and Fridays at 05:00 (e.g. "0 5 * * MON,WED,FRI"); modify the cron entry (the cron: field) accordingly so the job aligns with the rest of the release-branch matrix.
352-373:⚠️ Potential issue | 🟠 Major | ⚡ Quick winOperator nightly job is wired to the Helm test ref.
Line 372 uses
redhat-developer-rhdh-osd-gcp-helm-nightly, so the operator-nightly job does not execute an operator-specific test path.Suggested fix
steps: @@ test: - - ref: redhat-developer-rhdh-osd-gcp-helm-nightly + - ref: redhat-developer-rhdh-osd-gcp-operator-nightly workflow: redhat-developer-rhdh-osd-gcp-claim-cluster🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@ci-operator/config/redhat-developer/rhdh/redhat-developer-rhdh-release-1.10.yaml` around lines 352 - 373, The job is currently using the Helm test ref `redhat-developer-rhdh-osd-gcp-helm-nightly` in the test list, so the operator-nightly job (as: e2e-osd-gcp-operator-nightly, workflow: redhat-developer-rhdh-osd-gcp-claim-cluster) never runs an operator-specific test; replace the test ref with the operator test ref (e.g. `redhat-developer-rhdh-osd-gcp-operator-nightly`) so the job executes the operator-specific test path instead of the Helm test.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Duplicate comments:
In
`@ci-operator/config/redhat-developer/rhdh/redhat-developer-rhdh-release-1.10.yaml`:
- Line 53: Update the cron schedule for the release job by replacing the current
cron value "0 5 * * *" with a release-branch cadence that runs Mondays,
Wednesdays, and Fridays at 05:00 (e.g. "0 5 * * MON,WED,FRI"); modify the cron
entry (the cron: field) accordingly so the job aligns with the rest of the
release-branch matrix.
- Around line 352-373: The job is currently using the Helm test ref
`redhat-developer-rhdh-osd-gcp-helm-nightly` in the test list, so the
operator-nightly job (as: e2e-osd-gcp-operator-nightly, workflow:
redhat-developer-rhdh-osd-gcp-claim-cluster) never runs an operator-specific
test; replace the test ref with the operator test ref (e.g.
`redhat-developer-rhdh-osd-gcp-operator-nightly`) so the job executes the
operator-specific test path instead of the Helm test.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository YAML (base), Central YAML (inherited)
Review profile: CHILL
Plan: Enterprise
Run ID: 5c5aaca8-544c-4e0a-800a-34165d8866fc
⛔ Files ignored due to path filters (2)
ci-operator/jobs/redhat-developer/rhdh/redhat-developer-rhdh-release-1.10-periodics.yamlis excluded by!ci-operator/jobs/**ci-operator/jobs/redhat-developer/rhdh/redhat-developer-rhdh-release-1.10-presubmits.yamlis excluded by!ci-operator/jobs/**
📒 Files selected for processing (2)
ci-operator/config/redhat-developer/rhdh/redhat-developer-rhdh-release-1.10.yamlcore-services/prow/02_config/redhat-developer/rhdh/_prowconfig.yaml
|
@zdrapela: The following test failed, say
Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
|
/pj-rehearse ack |
|
@zdrapela: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
openshift-merge-bot
Bot
added
the
rehearsals-ack
|
/lgtm |
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: subhashkhileri, zdrapela The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
2 participants
Summary
redhat-developer/rhdhrelease-1.10mainwith release-branch adaptations (Slack channel, staggered cron schedules, no cleanup jobs,max_concurrencyon all jobs)release-1.9Rehearsed: https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/openshift_release/79332/rehearse-79332-periodic-ci-redhat-developer-rhdh-release-1.10-e2e-ocp-helm-nightly/2055180413776695296