Skip to content

Commit 855be1e

Browse files
ppxlppxl
authored
Merge pull request #8 from cloudogu/feature/plain_selenium_build_lib
Feature/plain selenium build lib resolve #7
2 parents d51aff8 + aab44e3 commit 855be1e

File tree

10 files changed

+446
-60
lines changed

10 files changed

+446
-60
lines changed

‎CHANGELOG.md‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
# Changelog
2+
All notable changes to this project will be documented in this file.
3+
4+
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
5+
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
6+
7+
## [Unreleased]
8+
9+
### Added
10+
- new lib function for running E2E tests with pure `Selenium` without the help of `Zalenium`
11+
12+
### Changed
13+
- signatures for `withMavenSettings` changed slightly so the maven central mirror can be configured
14+
- helper `generateZaleniumJobName` was renamed to `generateJobName`
15+
- `generateJobName` now uses `-` instead of `_` because of possible problems when used as DNS name

‎README.md‎

Lines changed: 109 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -3,15 +3,31 @@
33
zalenium-build-lib
44
=======
55

6-
Jenkins shared library that provides a step that runs a temporary [zalenium server](https://github.com/zalando/zalenium)
7-
in docker container, that records videos of selenium tests. The videos are archived at the Jenkins job by this library.
8-
9-
# Usage
6+
Jenkins shared library that provides a toolset for UI tests, featuring prominently [Zalenium](https://github.com/zalando/zalenium)
7+
among other tools:
8+
9+
1. Zalenium
10+
- Zalenium is a framework on top of the famous Selenium UI test framework but adds convinient video support.
11+
- a step that runs a temporary [Zalenium server](https://github.com/zalando/zalenium) in Docker container that records videos of selenium tests
12+
- the videos are automatically archived at the Jenkins job by this library
13+
1. [Selenium Grid](https://selenium.dev/documentation/en/grid/)
14+
- a step that runs a temporary Selenium Hub along with some Selenium worker nodes in Docker containers
15+
1. Docker network handling
16+
- a step to conveniently create Docker bridge networks, unique per Jenkins run
17+
- self-removing
18+
1. Truststore handling
19+
- a pair of steps to copy a Java truststore from build node to another
20+
- a good thing if you have a Java Docker container that needs to share the same set of certificates for network communication
21+
- self-removing
22+
23+
## [Working with Zalenium]
24+
25+
### Usage
1026

1127
In your `Jenkinsfile`
1228

1329
```groovy
14-
@Library('github.com/cloudogu/zalenium-build-lib@d8b74327') _
30+
@Library('github.com/cloudogu/zalenium-build-lib@versionTag') _
1531
1632
// ...
1733
stage('UI Test') {
@@ -34,7 +50,7 @@ withZalenium([ seleniumVersion : '3.14.0-p15' ]) { zaleniumIp ->
3450
}
3551
```
3652

37-
When the build is done you can download the videos from the jenkins build and watch them.
53+
When the build is done you can download the videos from the Jenkins build and watch them.
3854
Even more convenient: You could watch the videos in the browser directly. This in possible if either
3955

4056
* your Jenkins does not have a Content Security Policy (CSP), which is not recommended or
@@ -45,20 +61,103 @@ Even more convenient: You could watch the videos in the browser directly. This i
4561
* Or you start your Jenkins instance with
4662
`-Dhudson.model.DirectoryBrowserSupport.CSP="sandbox; default-src 'none'; img-src 'self'; style-src 'self'; media-src 'self';"`
4763

48-
## Docker Network creation
64+
### Attaching Test Reports to the Jenkins Build Run
65+
66+
The resulting videos aside, you may be interested in attaching the actual test report to the Jenkins build run. Due to
67+
the fact that the resulting test report varies under the chosen technology stack (read: different format and location)
68+
the test developer is the one responsible to attach the test report to the build run. This is done as usual with the
69+
[Jenkins pipeline step `archiveArtifacts`](https://jenkins.io/doc/pipeline/tour/tests-and-artifacts/).
70+
71+
72+
### Locking
73+
74+
Right now, only one Job can run Zalenium Tests at a time.
75+
This could be improved in the future.
76+
77+
### Why?
78+
79+
When multiple jobs executed we faced non-deterministic issues that the zalenium container was gone all of a sudden,
80+
connections were aborted or timed out.
81+
82+
So we implemented a lock before starting zalenium that can only be passed by one job at a time.
83+
It feels like this issue is gone now, but we're not sure if the lock was the proper fix.
84+
85+
### Locking requirements
86+
87+
We use the `lock` step of the [Lockable Resources Plugin](https://wiki.jenkins.io/display/JENKINS/Lockable+Resources+Plugin).
88+
89+
## [Working with Selenium]
90+
91+
For Selenium Grid to work it is vital that a Docker network is supplied (while Zalenium above may work with its own).
92+
If you don't want the hassle of creating a Docker network there is good news: The
93+
[Docker network step](#Docker Network creation) plays well into your cards.
94+
95+
### Usage
96+
97+
In your `Jenkinsfile`
98+
99+
```groovy
100+
@Library('github.com/cloudogu/zalenium-build-lib@versionTag') _
101+
102+
// ...
103+
stage('UI Test') {
104+
withDockerNetwork() { networkName ->
105+
withSelenium(networkName) { seleniumIp ->
106+
// Run your build, passing ${zaleniumIp} to it
107+
}
108+
}
109+
}
110+
```
111+
112+
Besides the Docker network name there are also a number of parameters that you may or may not pass to the step.
113+
114+
| Parameter | optional? | Default | Description |
115+
|-----------|-----------|---------|-------------|
116+
|seleniumHubImage | optional | 'selenium/hub' | the Selenium Docker container images to be used from hub.docker.com.|
117+
|seleniumVersion | optional | "3.141.59-zinc" | the Selenium Docker container image tag |
118+
|workerImageFF | optional | "selenium/node-firefox" | the Selenium Firefox worker node Docker container image. This matches automatically with the given Selenium hub version. |
119+
|workerImageChrome | optional | "selenium/node-chrome" | the Selenium Chrome worker node Docker container image. This matches automatically with the given Selenium hub version.|
120+
|firefoxWorkerCount| mandatory if no Chrome worker is used | 0 | the number of Firefox containers that should be started for the test |
121+
|chromeWorkerCount | mandatory if no Firefox worker is used | 0 | the number of Chrome containers that should be started for the test |
122+
|hubPortMapping | optional | 4444 | the port under which the Selenium Hub should be available |
123+
|debugSelenium | optional | false | set to `true` if you want your Jenkins run log to be filled with debug output |
124+
125+
Please note that `firefoxWorkerCount` AND `chromeWorkerCount` must not contain a value of zero. That would render the
126+
test unusable because there would no one to execute the tests. In this case the body will not be executed and a
127+
`ConfigurationException` will be thrown.
128+
129+
```groovy
130+
// example for starting Selenium with a certain parameter with a groovy map.
131+
// The rest of the parameters fallback to their defaults.
132+
withSelenium([ seleniumVersion : '3.14.0-p15' ]) { seleniumIp ->
133+
// Run your build, passing ${zaleniumIp} to it
134+
}
135+
```
136+
137+
Compared with the [Zalenium step](#Working with Zalenium) above, this Selenium step does not require locking.
138+
139+
### Attaching Test Reports to the Jenkins Build Run
140+
141+
The resulting videos aside, you may be interested in attaching the actual test report to the Jenkins build run. Due to
142+
the fact that the resulting test report varies under the chosen technology stack (read: different format and location)
143+
the test developer is the one responsible to attach the test report to the build run. This is done as usual with the
144+
[Jenkins pipeline step `archiveArtifacts`](https://jenkins.io/doc/pipeline/tour/tests-and-artifacts/).
145+
146+
## [Docker Network creation]
49147

50148
It is possible (although not necessary) to explicitly work with docker networks. This library supports the automatic creation and removal of a bridge network with a unique name.
51149

52-
### How
150+
### Usage
53151

54152
`withZalenium` accepts now an optional network name the Zalenium container can attach to the given network. Conveniently a docker network can be created with this pipeline step which provides the dynamically created network name.
55153

56154
```
57155
withDockerNetwork { networkName ->
58156
def yourConfig = [:]
59-
withZalenium(yourConfig, networkName) {}
157+
withZalenium(yourConfig, networkName) {
60158
docker.image("foo/bar:1.2.3").withRun("--network ${network}") {
61159
...
160+
}
62161
}
63162
64163
```
@@ -67,7 +166,7 @@ It is possible (although not necessary) to explicitly work with docker networks.
67166

68167
Often comes a Truststore into play while working with Jenkins and Java. Jenkins can accommodate necessary certificates in its truststore so Java applications like Maven (and others too!) can successfully interact with other parties, like download artifacts from artifact repositories or transport data over the network. Even so, it may be necessary to provide these Java applications with the right certificates when otherwise encrypted communication would fail without doing so.
69168

70-
## Simple Truststore pipeline
169+
### Simple Truststore pipeline
71170

72171
For such circumstances this library provides a small snippet. The global `truststore` variable ensures that any truststore files which are copied in the process are also removed at the end of both `copy` and `use` actions.
73172

@@ -106,23 +205,6 @@ node('anotherNode') {
106205
}
107206
```
108207

109-
## Locking
110-
111-
Right now, only one Job can run Zalenium Tests at a time.
112-
This could be improved in the future.
113-
114-
### Why?
115-
116-
When multiple jobs executed we faced non-deterministic issues that the zalenium container was gone all of a sudden,
117-
connections were aborted or timed out.
118-
119-
So we implemented a lock before starting zalenium that can only be passed by one job at a time.
120-
It feels like this issue is gone now, but we're not sure if the lock was the proper fix.
121-
122-
### How?
123-
124-
We use the `lock` step of the [Lockable Resources Plugin](https://wiki.jenkins.io/display/JENKINS/Lockable+Resources+Plugin).
125-
126208
# Troubleshooting
127209

128210
The logs of the zalenium container are stored in `$WORKSPACE/zalenium-docker.log`.

‎vars/helper.groovy‎

Lines changed: 16 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,19 @@
1-
String generateZaleniumJobName() {
2-
return "${JOB_BASE_NAME}_${BUILD_NUMBER}"
1+
/**
2+
* @deprecated Use generateJobName() instead.
3+
*
4+
* @return a string containing the generated name.
5+
*/
6+
@Deprecated String generateZaleniumJobName() {
7+
return generateJobName()
8+
}
9+
10+
/**
11+
* Generates an unique string based on the job name and the current build number.
12+
*
13+
* @return a string containing the generated name.
14+
*/
15+
String generateJobName() {
16+
return "${JOB_BASE_NAME}-${BUILD_NUMBER}"
317
}
418

519
String findHostName() {

‎vars/helper.txt‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,7 @@
1-
- generateZaleniumJobName() - creates a jobname unique to the current Jenkins job. For the 99th Jenkins build of the job
2-
"ACME" the job name would be "ACME_99".
1+
- generateZaleniumJobName() - deprecated - use 'generateJobName()' instead
2+
3+
- generateJobName() - creates a jobname unique to the current Jenkins job. For the 99th Jenkins build of the job
4+
"ACME" the job name would be "ACME-99".
35

46
- findHostName() - returns the host name of the current Jenkins node. This is usually needed for getting the FQDN of the
57
Cloudogu EcoSystem instance which runs the Jenkins master node.

‎vars/withDockerNetwork.groovy‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -18,12 +18,12 @@ void call(printDebugOutput = false, Closure inner) {
1818
}
1919
}
2020

21+
String generateJobName() {
22+
return "${JOB_BASE_NAME}-${BUILD_NUMBER}"
23+
}
24+
2125
void debugOut(boolean printDebugOutput, String logMessage) {
2226
if (printDebugOutput) {
2327
echo "DEBUG: " + logMessage
2428
}
25-
}
26-
27-
String generateJobName() {
28-
return "${JOB_BASE_NAME}_${BUILD_NUMBER}"
2929
}

‎vars/withMavenSettings.groovy‎

Lines changed: 79 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,5 @@
1+
final String defaultMavenCentralRepoName = "maven-central"
2+
13
/**
24
* Provides a simple custom maven settings.xml file to current working directory with Maven Central mirror in the
35
* current CES instance.
@@ -6,18 +8,24 @@
68
*
79
* <pre>
810
* nexusCreds = usernamePassword(credentialsId: 'jenkinsNexusServiceUser', passwordVariable: 'PASSWORD', usernameVariable: 'USERNAME')
9-
* withMavenSettings(nexusCreds, 'cesinstance.stage.host.tld', '/usr/share/maven') { settingsXml ->
11+
* withMavenSettings.settings(nexusCreds, 'cesinstance.stage.host.tld', '/usr/share/maven', 'maven-central/') { settingsXml ->
1012
* // do your maven call here
11-
* mvn "-s ${settingsXml} test"
12-
* } // settings.xml will be removed automatically
13+
* mvn "-s ${settingsXml} clean build"
14+
* // or from this library
15+
* withMavenSettings.mvn ${settingsXml}, "clean build"
16+
*} // settings.xml will be removed automatically
1317
* </pre>
1418
*
1519
* @param nexusCredentials Jenkins credentials which provide USERNAME and PASSWORD to an account which enables Nexus interaction
16-
* @param cesFQDN the full qualified domain name of the current CES instance, f. i. <code>cesinstance.stage.host.tld</code>
20+
* @param cesFQDN the full qualified domain name of the current CES instance, f. i. <code>cesinstance.stage.host.tld</code>. The prefix <code>https://</code> will be added automatically.
1721
* @param pathToLocalMavenRepository without the .m2 directory part, f. i. <code>/usr/share/maven</code>. The suffix <code>/.m2/repository</code> will be added automatically.
22+
* @param mirrorNexusPath relative path to a maven central mirror hosted inside a nexus instance. The prefix <code>/nexus/repository</code> will be added automatically. Defaults to "maven-central"
23+
* @param closure This closure will be executed when the Maven <code>settings.xml</code> file was successfully created.
1824
*/
19-
def settings(def nexusCredentials, String cesFQDN, String pathToLocalMavenRepository, Closure closure) {
25+
def settings(def nexusCredentials, String cesFQDN, String pathToLocalMavenRepository, String mirrorNexusPath=defaultMavenCentralRepoName, Closure closure) {
2026
echo "write settings.xml to ${pathToLocalMavenRepository}"
27+
echo "local Maven repository should be located at ${pathToLocalMavenRepository}/.m2/repository"
28+
2129
String settingsXml = "settings.xml"
2230
withCredentials([nexusCredentials]) {
2331
writeFile file: settingsXml, text: """
@@ -34,7 +42,7 @@ def settings(def nexusCredentials, String cesFQDN, String pathToLocalMavenReposi
3442
<mirror>
3543
<id>${cesFQDN}</id>
3644
<name>${cesFQDN} Central Mirror</name>
37-
<url>https://${cesFQDN}/nexus/repository/itzbundshared/</url>
45+
<url>https://${cesFQDN}/nexus/repository/${mirrorNexusPath}</url>
3846
<mirrorOf>central</mirrorOf>
3947
</mirror>
4048
</mirrors>
@@ -48,28 +56,89 @@ def settings(def nexusCredentials, String cesFQDN, String pathToLocalMavenReposi
4856
}
4957
}
5058

51-
def mvnWithSettings(def nexusCredentials, String cesFQDN, String mvnCallArgs) {
59+
/**
60+
* Provides a simple custom maven settings.xml file to current working directory with Maven Central mirror in the
61+
* current CES instance.
62+
* @param nexusCredentials Jenkins credentials which provide USERNAME and PASSWORD to an account which enables Nexus interaction
63+
* @param cesFQDN the full qualified domain name of the current CES instance, f. i. <code>cesinstance.stage.host.tld</code>. The prefix <code>https://</code> will be added automatically.
64+
* @param mirrorNexusPath relative path to a maven central mirror hosted inside a nexus instance. The prefix <code>/nexus/repository</code> will be added automatically. Defaults to "maven-central"
65+
* @param closure This closure will be executed when the Maven <code>settings.xml</code> file was successfully created.
66+
*/
67+
def settingsWithEnvHome(def nexusCredentials, String cesFQDN, String mirrorNexusPath=defaultMavenCentralRepoName, Closure closure) {
5268
def currentHome = env.HOME
53-
settings(nexusCredentials, cesFQDN, currentHome) { settingsXml ->
69+
settings(nexusCredentials, cesFQDN, currentHome, mirrorNexusPath, closure)
70+
}
71+
72+
/**
73+
* Creates a Maven <code>settings.xml</code> and executes Maven with the given arguments.
74+
* @param nexusCredentials Jenkins credentials which provide USERNAME and PASSWORD to an account which enables Nexus interaction
75+
* @param cesFQDN the full qualified domain name of the current CES instance, f. i. <code>cesinstance.stage.host.tld</code>. The prefix <code>https://</code> will be added automatically.
76+
* @param mirrorNexusPath relative path to a maven central mirror hosted inside a nexus instance. The prefix <code>/nexus/repository</code> will be added automatically. Defaults to "maven-central"
77+
* @param mvnCallArgs these arguments contain the Maven arguments
78+
*/
79+
def mvnWithSettings(def nexusCredentials, String cesFQDN, String mirrorNexusPath, String mvnCallArgs) {
80+
def currentHome = env.HOME
81+
settings(nexusCredentials, cesFQDN, currentHome, mirrorNexusPath) { settingsXml ->
5482
mvn settingsXml, mvnCallArgs
5583
}
5684
}
5785

5886
/**
59-
* This method extracts the Maven 3 installation from Jenkins and calls Maven with the given settings.xml and Maven arguments
87+
* Execute maven. This method extracts the Maven 3 installation from Jenkins and calls Maven with the given settings.xml and Maven arguments.
88+
*
89+
* Example call:
90+
*
91+
* <pre>
92+
* withMavenSettings.settings(nexusCreds, 'cesinstance.stage.host.tld', '/usr/share/maven', 'maven-central/') { settingsXml ->
93+
* // parameter parenthesis is optional
94+
* withMavenSettings.mvn settingsXml, "clean build"
95+
* // can be written also like this
96+
* withMavenSettings.mvn(settingsXml, "clean build")
97+
* }
98+
* </pre>
99+
*
100+
* @param settingsXml the path to a Maven <code>settings.xml</code> file. (mandatory)
101+
* @param mvnCallArgs these arguments contain the Maven arguments. Whether these arguments are optional or not depends on your <code>pom.xml</code>.
60102
*/
61103
def mvn(String settingsXml, String mvnCallArgs) {
62104
def mvnHome = tool 'M3'
63-
105+
64106
mvnWithHome(mvnHome, settingsXml, mvnCallArgs)
65107
}
66108

109+
/**
110+
* Creates a Maven <code>settings.xml</code> and executes Maven with the given arguments.
111+
* @param nexusCredentials Jenkins credentials which provide USERNAME and PASSWORD to an account which enables Nexus interaction
112+
* @param cesFQDN the full qualified domain name of the current CES instance, f. i. <code>cesinstance.stage.host.tld</code>. The prefix <code>https://</code> will be added automatically.
113+
* @param mvnHome the Maven home path
114+
* @param pathToLocalMavenRepository the path to the local Maven repository
115+
* @param mvnCallArgs these arguments contain the Maven arguments. Whether these arguments are optional or not depends on your <code>pom.xml</code>.
116+
*/
67117
def customMvnWithSettings(def nexusCredentials, String cesFQDN, String mvnHome, String pathToLocalMavenRepository, String mvnCallArgs) {
68118
settings(nexusCredentials, cesFQDN, pathToLocalMavenRepository) { settingsXml ->
69119
mvnWithHome(mvnHome, settingsXml, mvnCallArgs)
70120
}
71121
}
72122

123+
/**
124+
* Creates a Maven <code>settings.xml</code> and executes Maven with the given arguments.
125+
* @param nexusCredentials Jenkins credentials which provide USERNAME and PASSWORD to an account which enables Nexus interaction
126+
* @param cesFQDN the full qualified domain name of the current CES instance, f. i. <code>cesinstance.stage.host.tld</code>. The prefix <code>https://</code> will be added automatically.
127+
* @param mirrorNexusPath relative path to a maven central mirror hosted inside a nexus instance. The prefix <code>/nexus/repository</code> will be added automatically. Defaults to "maven-central"
128+
* @param mvnCallArgs these arguments contain the Maven arguments
129+
*/
130+
def customMvnWithSettings(def nexusCredentials, String cesFQDN, String mvnHome, String pathToLocalMavenRepository, String customMirror, String mvnCallArgs) {
131+
settings(nexusCredentials, cesFQDN, pathToLocalMavenRepository, customMirror) { settingsXml ->
132+
mvnWithHome(mvnHome, settingsXml, mvnCallArgs)
133+
}
134+
}
135+
136+
/**
137+
* Execute maven from the given Maven home.
138+
* @param mvnHome the path where Maven is installed. (mandatory)
139+
* @param settingsXml the path to a Maven <code>settings.xml</code> file. (mandatory)
140+
* @param mvnCallArgs these arguments contain the Maven arguments. Whether these arguments are optional or not depends on your <code>pom.xml</code>.
141+
*/
73142
def mvnWithHome(String mvnHome, String settingsXml, String mvnCallArgs) {
74143
sh "${mvnHome}/bin/mvn -s ${settingsXml} --batch-mode -V -U -e -Dsurefire.useFile=false ${mvnCallArgs}"
75144
}

0 commit comments

Comments
 (0)