You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Integrating with maven central and adding some simple metrics (#47)
* Integrating with maven central and adding some simple metrics
* Activating profile for oosrh
* forcing release test
* [maven-release-plugin] prepare release selenium-pom-framework-1.0.2
* [maven-release-plugin] prepare for next development iteration
* [maven-release-plugin] prepare release selenium-pom-framework-1.0.2
* Testing release
* Adding new deployment techniques
* Updating env var names
* Changes to pom and travis
* [maven-release-plugin] prepare release selenium-pom-framework-1.0.2
* [maven-release-plugin] prepare release selenium-pom-framework-1.0.2
* [maven-release-plugin] prepare release selenium-pom-framework-1.0.3
* [maven-release-plugin] prepare for next development iteration
* Bumping sonatype staging plugin version to latest
* Bumping stagung plugin
* [maven-release-plugin] prepare release selenium-pom-framework-1.0.4
* [maven-release-plugin] prepare for next development iteration
* Adding missing pom info
* [maven-release-plugin] prepare release selenium-pom-framework-1.0.6
* [maven-release-plugin] prepare for next development iteration
* [maven-release-plugin] prepare release selenium-pom-framework-1.0.7
* [maven-release-plugin] prepare for next development iteration
* README changes
* Adding contributors to pom. Updating README with latest version info and how tos for the instrumetnation and metrics bits. Adding configruation for metrics config.
* Adding new instrumetnation points
* Using graphite config from config file
* Fixing some codacy issues
* Fixing some codacy issues
Co-authored-by: travis-ci <[email protected]>
Copy file name to clipboardExpand all lines: README.md
+85-50Lines changed: 85 additions & 50 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,7 +8,7 @@
8
8
<!-- ALL-CONTRIBUTORS-BADGE:START - Do not remove or modify this section -->
9
9
<!-- ALL-CONTRIBUTORS-BADGE:END -->
10
10
11
-
**We've released version 1.0.0!**
11
+
**We've released version 1.0.0! And we've moved to Maven Central hosting and added instrumentation**
12
12
13
13
This toolkit is really a culmination of 10 years of working with teams building automated UI checks with Selenium. We've found time and again that we refactor or rewrite "frameworks" that wrap the entirity of the Selenium API (to little benefit). On top of this, we see so much brittle code that we wanted to put something out there that people could use and benefit from our experience. As a result, this toolkit provides an simple, lightweight (and well structured) way to launch, control and configure checks for Selenium/WebDriver in Java. It's a curation of all of the little libraries and fixes we've used over the years. So if you're at the start of your automation journey or you're just bored of writing the same "framework" over and over again, you're in the right place.
14
14
@@ -20,13 +20,12 @@ You'll find no junk (hopefully) here, but it's a fairly opinionated approach:
20
20
- We provide random test data generators from Mockneat (https://github.com/nomemory/mockneat) and UK Gov (https://github.com/dwp/nino-format-validation) as well as some of our own (for dates and stuff)
21
21
- There's an externalised (and overridable) configuration system, based on JSON
22
22
- We handle providing JUnit (we use version 5), Selenium and Hamcrest (for writing good assertions)
23
+
- The framework is instrumented to measure execution time of internals, and these metrics are available via JMX or Graphite.
23
24
- Our framework is unit and integration checked as well as monitored for code quality; we treat this as a production code base, not a second class citizen.
24
-
- We have a (developing) open roadmap in GitHub Issues (click the Issues tab to see what we have so far)
25
+
- We have a (developing) open road map in GitHub Issues (click the Issues tab to see what we have so far)
25
26
26
27
Coming soon:
27
28
- SauceLabs and BrowserStack configuration (issues #2 and #3).
28
-
- Instrumentation of the framework components to help visualise slow or under performing checks over time (issue #44).
29
-
- Published artefacts to Maven Central (so you can get the framework without a GitHub membership) (issue #27).
30
29
- Expand documentation and tutorials (issue #45)
31
30
- And lots more marked in the issues list: https://github.com/digital-delivery-academy/selenium-pom-framework/issues
32
31
@@ -47,71 +46,107 @@ submit an issue ticket.
47
46
## Documentation
48
47
49
48
- A full reference guide is here: https://github.com/digital-delivery-academy/selenium-pom-framework/wiki
50
-
- Technical reference documentation (javadocs) are here: https://digital-delivery-academy.github.io/selenium-pom-framework/javadoc-1.0.1
49
+
- Technical reference documentation (javadocs) are here: https://digital-delivery-academy.github.io/selenium-pom-framework/javadoc-1.0.7
51
50
52
51
## Example checks/project
53
52
54
53
See https://github.com/digital-delivery-academy/selenium-pom-example repository for an example of how to write Selenium checks using the Page Object Model. This repository uses this framework.
55
54
56
55
## Usage
57
56
58
-
Put this in your POM.xml.
57
+
This project is published to Maven Central, so you just need to put this in your POM.xml.
59
58
60
59
```
61
60
<dependency>
62
61
<groupId>uk.co.evoco</groupId>
63
62
<artifactId>selenium-pom-framework</artifactId>
64
-
<version>1.0.1</version>
63
+
<version>1.0.7</version>
65
64
</dependency>
66
65
```
67
66
68
-
You need to tell Maven how to access GitHub packages. So you need to edit your `settings.xml`.
67
+
**Note:** we used to publish releases to GitHub packages (which you can see if you click Packages above). This was restrictive because people HAD to have a GitHub account to use it, and had to mess around with their M2 settings. Maven Central was the right answer to this problem.
68
+
- You can see historic releases in the Packages tab above
69
+
- You can see all new releases on Maven Central here: https://search.maven.org/artifact/uk.co.evoco/selenium-pom-framework (we'll keep the Releases tab updated on this page, and we'll keep the latest version details in the README)
69
70
70
-
Typically you can do this by looking in `~/.m2/settings.xml`. You will need a GitHub Personal Access Token, which
71
-
you can do here (once you're logged in): https://github.com/settings/tokens
71
+
We skipped through a few minor versions setting up the Maven Central release, so there is no effective gap between 1.0.2 and 1.0.7.
72
72
73
-
An example configuration (`settings.xml`) would be:
73
+
## Instrumentation and metrics
74
+
The framework is instrumented with Dropwizard Metrics which has two reporters; JMX and Graphite.
75
+
76
+
To view the JMX reports, the JVM still needs to be running. You can access by running from a command line `jconsole` and then connecting to the JVM that is running the tests.
77
+
From `jconsole` you will need to open `MBeans` and then `metrics` to see the metrics that are being produced.
78
+
79
+
**Note:** We're probably going to remove JMX in coming releases because the nature of the framework is that it runs and then exits. When the container closes, the metrics go with it (unless we add something like Graphite to back it). We'll see. It'll be fine during debugging though. We'd recommend looking at the next bit about Graphite and Grafana.
80
+
81
+
Graphite is a much more powerful way to see performance of the framework over time, and to see where your tests are spending the most time. Here's some instructions about how to get started using Docker.
82
+
83
+
**Note:** Obviously you have to have `docker` installed. I'm sure you can Google that :)
84
+
85
+
We're going to setup `docker` containers for `graphite` which is essentially a cool way to track metrics over time and `grafana` which is a sexy way to visualise things and create some dashboards. We need to link these two containers so that they can access each other
86
+
and we're going to use the official containers from Graphite and Grafana themselves to get the latest versions. First up, we need a `docker` network:
87
+
88
+
`docker network create graphite_grafana`
89
+
90
+
Then we need to grab `graphite`:
91
+
92
+
```
93
+
docker run -d \
94
+
--name graphite \
95
+
--restart=always \
96
+
--net graphite_grafana \
97
+
-p 80:80 \
98
+
-p 2003-2004:2003-2004 \
99
+
-p 2023-2024:2023-2024 \
100
+
-p 8125:8125/udp \
101
+
-p 8126:8126 \
102
+
graphiteapp/graphite-statsd
103
+
```
104
+
105
+
And finally we need to grab `grafana`:
106
+
107
+
`docker run -d --name=grafana -p 3000:3000 --net graphite_grafana grafana/grafana`
108
+
109
+
Once all of these commands have finished if you run `docker ps` you should end up with something like the following:
110
+
111
+
```
112
+
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
113
+
acbce542bd1f graphiteapp/graphite-statsd "/entrypoint" 4 seconds ago Up 4 seconds 0.0.0.0:80->80/tcp, 0.0.0.0:2003-2004->2003-2004/tcp, 2013-2014/tcp, 8080/tcp, 0.0.0.0:2023-2024->2023-2024/tcp, 0.0.0.0:8126->8126/tcp, 8125/tcp, 0.0.0.0:8125->8125/udp graphite
114
+
ed8a7fcdbfc3 grafana/grafana "/run.sh" 2 minutes ago Up 2 minutes 0.0.0.0:3000->3000/tcp grafana
115
+
```
116
+
117
+
Now open up a browser window and get the two web interfaces loaded up:
118
+
119
+
Graphite: http://localhost:80
120
+
121
+
Grafana: http://localhost:3000
122
+
123
+
Grafana is where we're going to do most of our work now. Let's set up the data source first. From the home page in Grafana click "Add data source".
124
+
Then select "Graphite" from the list of sources.
125
+
126
+
Give an appropriate name (selenium-pom-framework for example). The following fields and values follow:
127
+
128
+
```
129
+
URL: http://graphite:80
130
+
```
131
+
132
+
Click `Save & Test`. Everything should go green, and Graphite and Grafana should now be connected. Now you can run tests and all of the metrics will be put out to Graphite and you can create dashboards to see
133
+
where your tests are going slowly.
134
+
135
+
There's a sample dashboard in this repo in `grafana-dashboard-examples/sample-dashboard.json` that you can import to get started.
136
+
137
+
If you're deploying this else where (i.e. not locally) you can configure connection details in the configuration object (see below).
0 commit comments