Merge pull request #280 from weizhouapache/e2e-update-readme

e2e: Add run-e2e command to test/e2e/README.md

Author: k8s-ci-robot

test/e2e/README.md

@@ -83,3 +83,90 @@ When this environment variable is set to *true* a 15s pause is inserted at the b
 - Launch the e2e test: *PAUSE_FOR_DEBUGGER_ATTACH=true JOB=MyTest make run-e2e*
 - Wait for console message: *Pausing 15s so you have a chance to attach a debugger to this process...*
 - Quickly attach your debugger to the e2e process (i.e., e2e.test)
+
+## CI/CD for e2e testing
+
+The community has set up a CI/CD pipeline using Jenkins for e2e testing.
+
+### How it works
+
+The CI/CD pipeline works as below
+
+- User triggers e2e testing by a Github PR comment, only repository OWNERS and a list of engineers are allowed;
+- A program monitors the PR comments of the Github repository, parses the comments and kick Jenkins jobs;
+- Jenkins creates a Apache CloudStack with specific version and hypervisor type, if needed;
+- Jenkins runs CAPC e2e testing with specific Kubernetes versions and images;
+- Jenkins posts the results of e2e testing as a Github PR comment, with the link of test logs.
+
+### How to use it
+
+Similar as other prow commands(see [here](https://prow.k8s.io/command-help?repo=kubernetes-sigs%2Fcluster-api-provider-cloudstack)), the e2e testing can be triggered by PR comment `/run-e2e`:
+
+```
+Usage: /run-e2e [-k Kubernetes_Version] [-c CloudStack_Version] [-h Hypervisor] [-i Template/Image]
+       [-f Kubernetes_Version_Upgrade_From] [-t Kubernetes_Version_Upgrade_To]
+```
+
+- Supported Kubernetes versions are: ['1.27.2', '1.26.5', '1.25.10', '1.24.14', '1.23.3', '1.22.6']. The default value is '1.27.2'.
+- Supported CloudStack versions are: ['4.18', '4.17', '4.16']. If it is not set, an existing environment will be used.
+- Supported hypervisors are: ['kvm', 'vmware', 'xen']. The default value is 'kvm'.
+- Supported templates are: ['ubuntu-2004-kube', 'rockylinux-8-kube']. The default value is 'ubuntu-2004-kube'.
+- By default it tests Kubernetes upgrade from version '1.26.5' to '1.27.2'.
+
+### Examples
+
+1. Examples of `/run-e2e` commands
+
+```
+/run-e2e
+/run-e2e -k 1.27.2 -h kvm -i ubuntu-2004-kube
+/run-e2e -k 1.27.2 -c 4.18 -h kvm -i ubuntu-2004-kube -f 1.26.5 -t 1.27.2
+```
+
+2. Example of test results
+```
+Test Results : (tid-126)
+Environment: kvm Rocky8(x3), Advanced Networking with Management Server Rocky8
+Kubernetes Version: v1.27.2
+Kubernetes Version upgrade from: v1.26.5
+Kubernetes Version upgrade to: v1.27.2
+CloudStack Version: 4.18
+Template: ubuntu-2004-kube
+E2E Test Run Logs: https://github.com/blueorangutan/capc-prs/releases/download/capc-pr-ci-cd/capc-e2e-artifacts-pr277-sl-126.zip
+
+[PASS] When testing Kubernetes version upgrades Should successfully upgrade kubernetes versions when there is a change in relevant fields
+[PASS] When testing subdomain Should create a cluster in a subdomain
+[PASS] When testing K8S conformance [Conformance] Should create a workload cluster and run kubetest
+[PASS] When testing app deployment to the workload cluster with slow network [ToxiProxy] Should be able to download an HTML from the app deployed to the workload cluster
+[PASS] When testing multiple CPs in a shared network with kubevip Should successfully create a cluster with multiple CPs in a shared network
+[PASS] When testing resource cleanup Should create a new network when the specified network does not exist
+[PASS] When testing app deployment to the workload cluster with network interruption [ToxiProxy] Should be able to create a cluster despite a network interruption during that process
+[PASS] When testing node drain timeout A node should be forcefully removed if it cannot be drained in time
+[PASS] When testing machine remediation Should replace a machine when it is destroyed
+[PASS] When testing with custom disk offering Should successfully create a cluster with a custom disk offering
+[PASS] When testing horizontal scale out/in [TC17][TC18][TC20][TC21] Should successfully scale machine replicas up and down horizontally
+[PASS] When testing MachineDeployment rolling upgrades Should successfully upgrade Machines upon changes in relevant MachineDeployment fields
+[PASS] with two clusters should successfully add and remove a second cluster without breaking the first cluster
+[PASS] When testing with disk offering Should successfully create a cluster with disk offering
+[PASS] When testing affinity group Should have host affinity group when affinity is pro
+[PASS] When testing affinity group Should have host affinity group when affinity is anti
+[PASS] When the specified resource does not exist Should fail due to the specified account is not found [TC4a]
+[PASS] When the specified resource does not exist Should fail due to the specified domain is not found [TC4b]
+[PASS] When the specified resource does not exist Should fail due to the specified control plane offering is not found [TC7]
+[PASS] When the specified resource does not exist Should fail due to the specified template is not found [TC6]
+[PASS] When the specified resource does not exist Should fail due to the specified zone is not found [TC3]
+[PASS] When the specified resource does not exist Should fail due to the specified disk offering is not found
+[PASS] When the specified resource does not exist Should fail due to the compute resources are not sufficient for the specified offering [TC8]
+[PASS] When the specified resource does not exist Should fail due to the specified disk offer is not customized but the disk size is specified
+[PASS] When the specified resource does not exist Should fail due to the specified disk offer is customized but the disk size is not specified
+[PASS] When the specified resource does not exist Should fail due to the public IP can not be found
+[PASS] When the specified resource does not exist When starting with a healthy cluster Should fail to upgrade worker machine due to insufficient compute resources
+[PASS] When the specified resource does not exist When starting with a healthy cluster Should fail to upgrade control plane machine due to insufficient compute resources
+[PASS] When testing app deployment to the workload cluster [TC1][PR-Blocking] Should be able to download an HTML from the app deployed to the workload cluster
+
+
+Ran 28 of 29 Specs in 10458.173 seconds
+SUCCESS! -- 28 Passed | 0 Failed | 0 Pending | 1 Skipped
+PASS
+
+```