Add release automation workflow.

This change adds the necessary gh workfows to automate releases for
minor releases.

Workflows are split into 3 components:
1. Prep
2. Tests
3. Release Creation

1. Prep
Prep workflow will validate compatibilty.yaml, and ensure the minor
branches (e.g. 1.3.x) are not already created. If branches already
exist, the Prep workflow will fail. If comptability.yaml does not
contain the minor version entry, the workflow will fail. If
compatibility.yaml contains the minor version entry, but comptaiblity.md
is not up to date, then workflow will create the necessary PR and exit
early, notifying the user to retry the Prep workflow once this PR is
merged and compatiblity docs are in sync.

If all pre-requisites are met, Prep workflow will create the necessary
minor branches in DSP and DSPO repositories, then continue to perform
all the necessary image builds for DSPO + DSP components. Once the
builds are done, the Prep workfow will generate the necessary follow up
PR to the minor release branch, updating the params.env with the SHAs of
the newly built images.

2. Tests
This is a placeholder workflow to add future tests required to perform a
stable release. These are ci tests that run on the PR created via Prep
workflow above. Once the CI tests pass, the PR will need to be manually
merged, the workflow does not auto merge PRs (nor does it auto push
commits to any branches).

3. Release Creation
Once the PR from (1) is merged. A trigger workflow is launched, this
workflow exists to trigger the "Create Release" workflow. The trigger
workflow will package required information that is pulled from the PR
body (in the form of a yaml) and will upload it to GH workflow cache,
for the "Create Release" workflow to utilize. Once the trigger workflow
completes, the "Create Release" worklfow will begin.

The "Create Release" workflow Validates the triggering PR from (1), this
step is to prevent unauthorized entities from triggering this workflow.
Validation confirms the PR info being passed into this workflow
originates from a PR authored by a user in the "approvers" list in DSPO
repo's OWNERS file. Once validated, "Create Release" workflow will
create releases in DSP/DSPO, providing links to these releases in the
workflow summary, as well as a PR comment to the PR created in (1).

Signed-off-by: Humair Khan <[email protected]>

Author: HumairAK

.github/scripts/release_create/create_tag_release.sh

@@ -0,0 +1,32 @@
+#!/usr/bin/env bash
+
+set -ex
+
+echo "Create a tag release for ${TARGET_VERSION_TAG} in ${REPOSITORY}"
+
+RELEASE_REPO_DIR=$(dirname ${WORKING_DIR})/repo_dir
+git clone \
+  --depth=1 \
+  --branch=${RELEASE_BRANCH} \
+  https://${GH_USER_NAME}:${GH_TOKEN}@github.com/${REPOSITORY} \
+  ${RELEASE_REPO_DIR}
+cd ${RELEASE_REPO_DIR}
+
+gh release create ${TARGET_VERSION_TAG} --target ${RELEASE_BRANCH} --generate-notes --notes-start-tag ${PREVIOUS_VERSION_TAG}
+
+cat <<EOF >> /tmp/release-notes.md
+
+This is a release comprising of multiple repos:
+* DSP component for ${TARGET_VERSION_TAG} can be found [here](https://github.com/${GH_ORG}/data-science-pipelines/releases/tag/${TARGET_VERSION_TAG})
+* DSPO component for ${TARGET_VERSION_TAG} can be found [here](https://github.com/${GH_ORG}/data-science-pipelines-operator/releases/tag/${TARGET_VERSION_TAG})
+
+Version Table for components can be found [here](https://github.com/${GH_ORG}/data-science-pipelines-operator/blob/main/docs/release/compatibility.md)
+EOF
+
+echo "$(gh release view ${TARGET_VERSION_TAG} --json body --jq .body)" >> /tmp/release-notes.md
+
+echo "Release notes to be created:"
+cat /tmp/release-notes.md
+
+gh release edit ${TARGET_VERSION_TAG} --notes-file /tmp/release-notes.md
+rm /tmp/release-notes.md

.github/scripts/release_create/notify.sh

@@ -0,0 +1,17 @@
+#!/usr/bin/env bash
+
+set -ex
+
+cat <<EOF >> /tmp/body-file.txt
+Release created successfully:
+
+https://github.com/${GH_ORG}/data-science-pipelines-operator/releases/tag/${TARGET_VERSION_TAG}
+
+https://github.com/${GH_ORG}/data-science-pipelines/releases/tag/${TARGET_VERSION_TAG}
+EOF
+
+gh pr comment ${PR_NUMBER} --body-file /tmp/body-file.txt
+
+echo "::notice:: DSPO Release: https://github.com/${GH_ORG}/data-science-pipelines-operator/releases/tag/${TARGET_VERSION_TAG}"
+echo "::notice:: DSP Release: https://github.com/${GH_ORG}/data-science-pipelines/releases/tag/${TARGET_VERSION_TAG}"
+echo "::notice:: Feedback sent to PR."

.github/scripts/release_create/validate_pr.sh

@@ -0,0 +1,23 @@
+#!/usr/bin/env bash
+
+set -ex
+
+echo "::notice:: Performing Release PR Validation for: ${PR_NUMBER}"
+
+# Retrive PR Author:
+PR_AUTHOR=$(gh pr view ${PR_NUMBER} --json author -q .author.login)
+
+echo "Current OWNERS:"
+cat ./OWNERS
+
+echo "::notice:: Checking if PR author ${PR_AUTHOR} is DSPO Owner..."
+
+is_owner=$(cat ./OWNERS | var=${PR_AUTHOR} yq '[.approvers] | contains([env(var)])')
+if [[ $is_owner == "false" ]]; then
+  echo "::error:: PR author ${PR_AUTHOR} is not an approver in OWNERS file. Only approvers can create releases."
+  exit 1
+fi
+
+echo "::notice:: PR author ${PR_AUTHOR} is an approver in DSPO OWNERS."
+
+echo "::notice:: Validation successful."

.github/scripts/release_create/vars.sh

@@ -0,0 +1,16 @@
+#!/usr/bin/env bash
+
+set -ex
+
+cat ./config.yaml
+target_version_tag=$(yq .target_version_tag ./config.yaml)
+previous_version_tag=$(yq .previous_release_tag ./config.yaml)
+release_branch=$(yq .release_branch ./config.yaml)
+odh_org=$(yq .odh_org ./config.yaml)
+pr_number=$(cat ./pr_number)
+
+echo "pr_number=${pr_number}" >> $GITHUB_OUTPUT
+echo "target_version_tag=${target_version_tag}" >> $GITHUB_OUTPUT
+echo "previous_version_tag=${previous_version_tag}" >> $GITHUB_OUTPUT
+echo "release_branch=${release_branch}" >> $GITHUB_OUTPUT
+echo "odh_org=${odh_org}" >> $GITHUB_OUTPUT

.github/scripts/release_prep/create_branches.sh

@@ -0,0 +1,24 @@
+#!/usr/bin/env bash
+
+set -ex
+
+echo "Cut branch ${MINOR_RELEASE_BRANCH} from main/master"
+
+echo "Current branches in ${DSPO_REPOSITORY_FULL}"
+git branch -r
+
+git checkout -B ${MINOR_RELEASE_BRANCH}
+git push origin ${MINOR_RELEASE_BRANCH}
+echo "::notice:: Created DSPO ${MINOR_RELEASE_BRANCH} branch"
+
+echo "Current branches in ${DSP_REPOSITORY_FULL}"
+DSP_DIR=$(dirname ${WORKING_DIR})/data-science-pipelines
+git clone \
+  --depth=1 \
+  --branch=master \
+  https://${GH_USER_NAME}:${GH_TOKEN}@github.com/${DSP_REPOSITORY_FULL} \
+  ${DSP_DIR}
+cd ${DSP_DIR}
+git checkout -B ${MINOR_RELEASE_BRANCH}
+git push origin ${MINOR_RELEASE_BRANCH}
+echo "::notice:: Created DSP ${MINOR_RELEASE_BRANCH} branch"

.github/scripts/release_prep/generate_pr.sh

@@ -0,0 +1,66 @@
+#!/usr/bin/env bash
+
+# Note: The yaml in the body of the PR is used to feed inputs into the release workflow
+# since there's no easy way to communicate information between the pr closing, and then triggering the
+# release creation workflow.
+# Therefore, take extra care when adding new code blocks in the PR body, or updating the existing one.
+# Ensure any changes are compatible with the release_create workflow.
+
+set -ex
+set -o pipefail
+
+echo "Retrieve the sha images from the resulting workflow (check quay.io for the digests)."
+echo "Using [release-tools] generate a params.env and submit a new pr to vx.y+1.**x** branch."
+echo "For images pulled from registry, ensure latest images are upto date"
+
+BRANCH_NAME="release-${TARGET_RELEASE}"
+git config --global user.email "${GH_USER_EMAIL}"
+git config --global user.name "${GH_USER_NAME}"
+git remote add ${GH_USER_NAME} https://${GH_USER_NAME}:${GH_TOKEN}@github.com/${GH_USER_NAME}/${DSPO_REPOSITORY}.git
+git checkout -B ${BRANCH_NAME}
+
+echo "Created branch: ${BRANCH_NAME}"
+
+python ./scripts/release/release.py params --quay_org ${QUAY_ORG} --tag ${MINOR_RELEASE_TAG} --out_file ./config/base/params.env \
+  --override="IMAGES_OAUTHPROXY=registry.redhat.io/openshift4/ose-oauth-proxy@sha256:ab112105ac37352a2a4916a39d6736f5db6ab4c29bad4467de8d613e80e9bb33"
+
+git add .
+git commit -m "Generate params for ${TARGET_RELEASE}"
+git push ${GH_USER_NAME} $BRANCH_NAME -f
+
+# Used to feed inputs to release creation workflow.
+# target_version is used as the GH TAG
+tmp_config="/tmp/body-config.txt"
+body_txt="/tmp/body-text.txt"
+cp $CONFIG_TEMPLATE $tmp_config
+
+var=${GH_ORG} yq -i '.odh_org=env(var)' $tmp_config
+var=${MINOR_RELEASE_BRANCH} yq -i '.release_branch=env(var)' $tmp_config
+var=${MINOR_RELEASE_TAG} yq -i '.target_version_tag=env(var)' $tmp_config
+var=${PREVIOUS_RELEASE_TAG} yq -i '.previous_release_tag=env(var)' $tmp_config
+
+
+cat <<"EOF" > $body_txt
+This is an automated PR to prep Data Science Pipelines Operator for release.
+```yaml
+<CONFIG_HERE>
+```
+EOF
+
+sed -i "/<CONFIG_HERE>/{
+    s/<CONFIG_HERE>//g
+    r ${tmp_config}
+}" $body_txt
+
+
+pr_url=$(gh pr create \
+  --repo https://github.com/${DSPO_REPOSITORY_FULL} \
+  --body-file $body_txt \
+  --title "Release ${MINOR_RELEASE_TAG}" \
+  --head "${GH_USER_NAME}:$BRANCH_NAME" \
+  --label "release-automation" \
+  --base "${MINOR_RELEASE_BRANCH}")
+
+
+
+echo "::notice:: PR successfully created: ${pr_url}"

.github/scripts/release_prep/prereqs.sh

@@ -0,0 +1,78 @@
+#!/usr/bin/env bash
+
+set -ex
+
+check_branch_exists(){
+  branch_exists=$(git ls-remote --heads https://github.com/${1}.git refs/heads/${2})
+  echo "Checking for existence of branch ${2} in GH Repo ${1}"
+  if [[ $branch_exists ]]; then
+    echo "::error:: Branch ${2} already exist in GH Repo ${1}"
+    exit 1
+  fi
+  echo "::notice:: Confirmed Branch ${2} does not exist in GH Repo ${1}"
+}
+
+check_branch_exists ${DSPO_REPOSITORY_FULL} ${MINOR_RELEASE_BRANCH}
+check_branch_exists ${DSP_REPOSITORY_FULL} ${MINOR_RELEASE_BRANCH}
+
+echo "Ensure compatibility.yaml is upto date, and generate a new compatibility.md. Use [release-tools] to accomplish this"
+
+BRANCH_NAME="compatibility-doc-generate-${TARGET_RELEASE}"
+
+git config --global user.email "${GH_USER_EMAIL}"
+git config --global user.name "${GH_USER_NAME}"
+git remote add ${GH_USER_NAME} https://${GH_USER_NAME}:${GH_TOKEN}@github.com/${GH_USER_NAME}/${DSPO_REPOSITORY}.git
+git checkout -B ${BRANCH_NAME}
+
+echo "Created branch: ${BRANCH_NAME}"
+echo "Checking if compatibility.yaml contains ${TARGET_RELEASE} release...."
+
+contains_rel=$(cat docs/release/compatibility.yaml | rel=${MINOR_RELEASE_WILDCARD} yq '[.[].dsp] | contains([env(rel)])')
+
+if [[ "$contains_rel" == "false" ]]; then
+
+cat <<EOF >> /tmp/error.txt
+compatibility.yaml has NOT been updated with target release.
+
+Please add ${MINOR_RELEASE_WILDCARD} dsp row in compatibility.yaml,
+
+then regenerate the compatibility.md by following the instructions here:
+https://github.com/opendatahub-io/data-science-pipelines-operator/tree/main/scripts/release#compatibility-doc-generation
+EOF
+
+echo ::error::$(cat /tmp/error.txt)
+exit 1
+
+fi
+
+echo "::notice:: Confirmed existence of ${MINOR_RELEASE_BRANCH} in compatibility.yaml."
+
+echo "Confirming that compatibility.md is upto date."
+python ./scripts/release/release.py version_doc --input_file docs/release/compatibility.yaml --out_file docs/release/compatibility.md
+
+git status
+
+prereqs_successful=true
+
+if [[ `git status --porcelain` ]]; then
+  echo "::notice:: Compatibility.md is not up to date with Compatibility.yaml, creating pr to synchronize."
+
+  git add .
+  git commit -m "Update DSPO to $TARGET_RELEASE"
+  git push ${GH_USER_NAME} $BRANCH_NAME -f
+  gh pr create \
+    --repo https://github.com/${DSPO_REPOSITORY_FULL} \
+    --body "This is an automated PR to update Data Science Pipelines Operator version compatibility doc." \
+    --title "Update DSP version compatibility doc." \
+    --head "${GH_USER_NAME}:$BRANCH_NAME" \
+    --base "main"
+
+  echo "::notice:: PR to update compatibility doc has been created, please re-run this workflow once this PR is merged."
+  prereqs_successful=false
+else
+  echo "::notice:: Compatibility.md doc is up to date with Compatibility.yaml, continuing with workflow..."
+fi
+
+# Save step outputs
+echo "prereqs_successful=${prereqs_successful}"
+echo "prereqs_successful=${prereqs_successful}" >> $GITHUB_OUTPUT

.github/scripts/release_prep/templates/config.yaml

@@ -0,0 +1,4 @@
+odh_org: placeholder
+release_branch: placeholder
+target_version_tag: placeholder
+previous_release_tag: placeholder

.github/scripts/release_trigger/upload-data.sh

@@ -0,0 +1,18 @@
+#!/usr/bin/env bash
+set -ex
+set -o pipefail
+
+mkdir -p ./pr
+
+cat <<EOF >> /tmp/body-file-raw.txt
+${PR_BODY}
+EOF
+
+sed -n '/^```yaml/,/^```/ p' < /tmp/body-file-raw.txt | sed '/^```/ d' > ./pr/config.yaml
+echo Parsed config from PR body:
+yq ./pr/config.yaml
+
+# Also store pr details
+echo ${PR_NUMBER} >> ./pr/pr_number
+echo ${PR_STATE} >> ./pr/pr_state
+echo ${PR_HEAD_SHA} >> ./pr/head_sha

.github/scripts/tests/tests.sh

@@ -0,0 +1,5 @@
+#!/usr/bin/env bash
+
+set -ex
+
+echo "Perform any tests on the branch, confirm stability. If issues are found, they should be corrected in `main/master` and be cherry-picked into this branch."