-
Notifications
You must be signed in to change notification settings - Fork 78
KUBESAW-230: improve pairing mechanism #1113
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
rsoaresd
wants to merge
3
commits into
codeready-toolchain:master
Choose a base branch
from
rsoaresd:test-pairing
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -17,20 +17,8 @@ endif | |
| get-ksctl-and-install: | ||
| ifeq ($(strip $(KSCTL_REPO_PATH)),) | ||
| ifneq ($(CI_DISABLE_PAIRING),true) | ||
| ifeq ($(shell jq -r '.refs[0].org' <<< $${CLONEREFS_OPTIONS} 2>/dev/null || true | tr -d '[:space:]'),codeready-toolchain) | ||
| $(eval AUTHOR_LINK = $(shell jq -r '.refs[0].pulls[0].author_link' <<< $${CLONEREFS_OPTIONS} | tr -d '[:space:]')) | ||
| $(eval BRANCH_NAME := $(shell jq -r '.refs[0].pulls[0].head_ref' <<< $${CLONEREFS_OPTIONS} | tr -d '[:space:]')) | ||
| @echo "using author link ${AUTHOR_LINK}" | ||
| @echo "detected branch ${BRANCH_NAME}" | ||
| # check if a branch with the same ref exists in the user's fork of ksctl repo | ||
| @echo "branches of ${AUTHOR_LINK}/ksctl - checking if there is a branch ${BRANCH_NAME} we could pair with." | ||
| curl ${AUTHOR_LINK}/ksctl.git/info/refs?service=git-upload-pack --output - | ||
| $(eval REMOTE_KSCTL_BRANCH := $(shell curl ${AUTHOR_LINK}/ksctl.git/info/refs?service=git-upload-pack --output - 2>/dev/null | grep -a "refs/heads/${BRANCH_NAME}$$" | awk '{print $$2}')) | ||
| # check if the branch with the same name exists, if so then merge it with master and use the merge branch, if not then use master | ||
| @$(MAKE) clone-ksctl-and-pair REMOTE_KSCTL_BRANCH=${REMOTE_KSCTL_BRANCH} AUTHOR_LINK=${AUTHOR_LINK} | ||
| else | ||
| @echo "Either not running in openshift-ci or not in the codeready-toolchain org - no pairing" | ||
| endif | ||
| ${PAIRING_EXEC} pair --clone-dir /tmp/ksctl --organization kubesaw --repository ksctl | ||
| $(eval KSCTL_REPO_PATH := /tmp/ksctl) | ||
| else | ||
| @echo "Pairing explicitly disabled" | ||
| endif | ||
|
|
@@ -39,28 +27,6 @@ else | |
| endif | ||
| @$(MAKE) install-ksctl KSCTL_REPO_PATH=${KSCTL_REPO_PATH} REMOTE_KSCTL_BRANCH=${REMOTE_KSCTL_BRANCH} | ||
|
|
||
| clone-ksctl-and-pair: | ||
| ifneq ($(strip $(REMOTE_KSCTL_BRANCH)),) | ||
| @echo "Branch ref of the user's fork to be used for pairing: \"${REMOTE_KSCTL_BRANCH}\"" | ||
| # define temp dir | ||
| $(eval KSCTL_REPO_PATH := /tmp/ksctl) | ||
| # delete to have clear environment | ||
| rm -rf ${KSCTL_REPO_PATH} | ||
|
|
||
| git config --global user.email "[email protected]" | ||
| git config --global user.name "KubeSaw" | ||
| # clone | ||
| git clone --depth=1 https://github.com/kubesaw/ksctl.git ${KSCTL_REPO_PATH} | ||
| # add the user's fork as remote repo | ||
| git --git-dir=${KSCTL_REPO_PATH}/.git --work-tree=${KSCTL_REPO_PATH} remote add external ${AUTHOR_LINK}/ksctl.git | ||
| # fetch the branch | ||
| git --git-dir=${KSCTL_REPO_PATH}/.git --work-tree=${KSCTL_REPO_PATH} fetch external ${REMOTE_KSCTL_BRANCH} | ||
| # merge the branch with master | ||
| git --git-dir=${KSCTL_REPO_PATH}/.git --work-tree=${KSCTL_REPO_PATH} merge --allow-unrelated-histories --no-commit FETCH_HEAD | ||
| else | ||
| @echo "No branch for pairing found" | ||
| endif | ||
|
|
||
| install-ksctl: | ||
| ifneq ($(strip $(KSCTL_REPO_PATH)$(REMOTE_KSCTL_BRANCH)),) | ||
| @echo "Installing ksctl from directory $(or ${KSCTL_REPO_PATH}, /tmp/ksctl)" | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -9,18 +9,13 @@ fi | |
| WAS_ALREADY_PAIRED_FILE=/tmp/toolchain_e2e_already_paired | ||
|
|
||
| get_repo() { | ||
| PAIRED=false | ||
| if [[ -z ${PROVIDED_REPOSITORY_PATH} ]]; then | ||
| REPOSITORY_PATH="/tmp/codeready-toolchain/${REPOSITORY_NAME}" | ||
| rm -rf ${REPOSITORY_PATH} | ||
| # clone | ||
| git clone https://github.com/codeready-toolchain/${REPOSITORY_NAME}.git ${REPOSITORY_PATH} | ||
|
|
||
| pair_repo_if_needed | ||
|
|
||
| ${PAIRING_EXEC} pair --clone-dir ${REPOSITORY_PATH} --organization codeready-toolchain --repository ${REPOSITORY_NAME} | ||
| else | ||
| REPOSITORY_PATH=${PROVIDED_REPOSITORY_PATH} | ||
| fi | ||
|
|
||
| } | ||
|
|
||
| set_tags() { | ||
|
|
@@ -47,117 +42,11 @@ set_tags() { | |
| push_image() { | ||
| GIT_COMMIT_ID=$(git --git-dir=${REPOSITORY_PATH}/.git --work-tree=${REPOSITORY_PATH} rev-parse --short HEAD) | ||
| IMAGE_LOC=quay.io/codeready-toolchain/${REPOSITORY_NAME}:${GIT_COMMIT_ID} | ||
| if is_provided_or_paired; then | ||
| IMAGE_BUILDER=${IMAGE_BUILDER:-"podman"} | ||
| make -C ${REPOSITORY_PATH} ${IMAGE_BUILDER}-push QUAY_NAMESPACE=${QUAY_NAMESPACE} IMAGE_TAG=${TAGS} | ||
| IMAGE_LOC=quay.io/${QUAY_NAMESPACE}/${REPOSITORY_NAME}:${TAGS} | ||
| fi | ||
| IMAGE_BUILDER=${IMAGE_BUILDER:-"podman"} | ||
| make -C ${REPOSITORY_PATH} ${IMAGE_BUILDER}-push QUAY_NAMESPACE=${QUAY_NAMESPACE} IMAGE_TAG=${TAGS} | ||
| IMAGE_LOC=quay.io/${QUAY_NAMESPACE}/${REPOSITORY_NAME}:${TAGS} | ||
| } | ||
|
|
||
| is_provided_or_paired() { | ||
| [[ -n ${PROVIDED_REPOSITORY_PATH} ]] || [[ ${PAIRED} == true ]] | ||
| } | ||
|
|
||
| pair_repo_if_needed() { | ||
| if [[ -n ${GITHUB_ACTIONS} ]]; then | ||
| PR_REPO_NAME=${GITHUB_REPOSITORY##*/} | ||
| else | ||
| PR_REPO_NAME=$(echo ${REPO_NAME} | sed 's/"//g') | ||
| fi | ||
|
|
||
|
|
||
| if [[ -n "${CI}${CLONEREFS_OPTIONS}" ]] && [[ ${PR_REPO_NAME} == "toolchain-e2e" ]]; then | ||
| if [[ -n ${CLONEREFS_OPTIONS} ]]; then | ||
| # get branch ref of the fork the PR was created from | ||
| AUTHOR_LINK=$(jq -r '.refs[0].pulls[0].author_link' <<< ${CLONEREFS_OPTIONS} | tr -d '[:space:]') | ||
| PULL_PULL_SHA=${PULL_PULL_SHA:-$(jq -r '.refs[0].pulls[0].sha' <<< ${CLONEREFS_OPTIONS} | tr -d '[:space:]')} | ||
| echo "using author link ${AUTHOR_LINK}" | ||
| echo "using pull sha ${PULL_PULL_SHA}" | ||
| # get branch ref of the fork the PR was created from | ||
| REPO_URL=${AUTHOR_LINK}/toolchain-e2e | ||
| echo "branches of ${REPO_URL} - trying to detect the branch name we should use for pairing." | ||
| curl ${REPO_URL}.git/info/refs?service=git-upload-pack --output - | ||
| GET_BRANCH_NAME=$(curl ${REPO_URL}.git/info/refs?service=git-upload-pack --output - 2>/dev/null | grep -a ${PULL_PULL_SHA} || true) | ||
| if [[ $(echo ${GET_BRANCH_NAME} | wc -l) > 1 ]]; then \ | ||
| echo "################################### ERROR DURING THE E2E TEST SETUP ################################### | ||
| There were found more branches with the same latest commit '${PULL_PULL_SHA}' in the repo ${REPO_URL} - see: | ||
|
|
||
| ${GET_BRANCH_NAME} | ||
|
|
||
| It's not possible to detect the correct branch this PR is made for. | ||
| Please delete the unrelated branch from your fork and rerun the e2e tests. | ||
| Note: If you have already deleted the unrelated branch from your fork, it can take a few hours before the | ||
| github api is updated so the e2e tests may still fail with the same error until then. | ||
| ##########################################################################################################" | ||
| exit 1 | ||
| fi | ||
| BRANCH_REF=$(echo ${GET_BRANCH_NAME} | awk '{print $2}') | ||
| echo "detected branch ref ${BRANCH_REF}" | ||
| # retrieve the branch name | ||
| BRANCH_NAME=$(echo ${BRANCH_REF} | awk -F'/' '{print $3}') | ||
| else | ||
| AUTHOR_LINK=https://github.com/${AUTHOR} | ||
| BRANCH_REF=refs/heads/${GITHUB_HEAD_REF} | ||
| BRANCH_NAME=${GITHUB_HEAD_REF} | ||
| REPO_URL=${AUTHOR_LINK}/toolchain-e2e | ||
| fi | ||
|
|
||
| if [[ -n "${BRANCH_REF}" ]]; then \ | ||
| # check if a branch with the same ref exists in the user's fork of ${REPOSITORY_NAME} repo | ||
| echo "branches of ${AUTHOR_LINK}/${REPOSITORY_NAME} - checking if there is a branch ${BRANCH_REF} we could pair with." | ||
| curl ${AUTHOR_LINK}/${REPOSITORY_NAME}.git/info/refs?service=git-upload-pack --output - | ||
| REMOTE_E2E_BRANCH=$(curl ${AUTHOR_LINK}/${REPOSITORY_NAME}.git/info/refs?service=git-upload-pack --output - 2>/dev/null | grep -a "${BRANCH_REF}$" | awk '{print $2}') | ||
| echo "branch ref of the user's fork: \"${REMOTE_E2E_BRANCH}\" - if empty then not found" | ||
| # check if the branch with the same name exists, if so then merge it with master and use the merge branch, if not then use master \ | ||
| if [[ -n "${REMOTE_E2E_BRANCH}" ]]; then \ | ||
| if [[ -f ${WAS_ALREADY_PAIRED_FILE} ]]; then \ | ||
| echo "#################################### ERROR WHILE TRYING TO PAIR PRs #################################### | ||
| There was an error while trying to pair this e2e PR with ${AUTHOR_LINK}/${REPOSITORY_NAME}@${BRANCH_REF} | ||
| The reason is that there was already detected a branch from another repo this PR could be paired with - see: | ||
|
|
||
| $(cat ${WAS_ALREADY_PAIRED_FILE}) | ||
|
|
||
| It's not possible to pair a PR with multiple branches from other repositories. | ||
| Please delete one of the branches from your fork and rerun the e2e tests | ||
| Note: If you have already deleted one of the branches from your fork, it can take a few hours before the | ||
| github api is updated so the e2e tests may still fail with the same error until then. | ||
| ##########################################################################################################" | ||
| exit 1 | ||
| fi | ||
|
|
||
| git config --global user.email "[email protected]" | ||
| git config --global user.name "KubeSaw" | ||
|
|
||
| echo -e "repository: ${AUTHOR_LINK}/${REPOSITORY_NAME} \nbranch: ${BRANCH_NAME}" > ${WAS_ALREADY_PAIRED_FILE} | ||
| # add the user's fork as remote repo | ||
| git --git-dir=${REPOSITORY_PATH}/.git --work-tree=${REPOSITORY_PATH} remote add external ${AUTHOR_LINK}/${REPOSITORY_NAME}.git | ||
| # fetch the branch | ||
| git --git-dir=${REPOSITORY_PATH}/.git --work-tree=${REPOSITORY_PATH} fetch external ${BRANCH_REF} | ||
|
|
||
| echo "!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!" | ||
| echo "!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! WARNING !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!" | ||
| echo "!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!" | ||
| echo "" | ||
| echo "The following command will try to merge the paired PR using fast-forward way." | ||
| echo "If the command fails, then it means that the paired PR https://github.com/codeready-toolchain/${REPOSITORY_NAME}/ from branch ${BRANCH_NAME}" | ||
| echo "is not up-to-date with master and the fast-forward merge cannot be performed." | ||
| echo "If this happens, then rebase the PR with the latest changes from master and rerun this GH Actions build (or comment /retest in this PR)." | ||
| echo " https://github.com/codeready-toolchain/${REPOSITORY_NAME}/pulls?q=head%3A${BRANCH_NAME}" | ||
| echo "" | ||
| echo "!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!" | ||
| echo "!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!" | ||
| # merge the branch with master using fast-forward | ||
| git --git-dir=${REPOSITORY_PATH}/.git --work-tree=${REPOSITORY_PATH} merge --ff-only FETCH_HEAD | ||
| # print information about the last three commits, so we know what was merged plus some additional context/history | ||
| git --git-dir=${REPOSITORY_PATH}/.git --work-tree=${REPOSITORY_PATH} log --ancestry-path HEAD~3..HEAD | ||
|
|
||
| PAIRED=true | ||
| fi | ||
| fi | ||
| fi | ||
| } | ||
|
|
||
|
|
||
| install_operator() { | ||
| DISPLAYNAME=$(echo ${OPERATOR_NAME} | tr '-' ' ' | awk '{for (i=1;i<=NF;i++) $i=toupper(substr($i,1,1)) substr($i,2)} 1') | ||
|
|
||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
just temporary until codeready-toolchain/toolchain-cicd#135 is merged