Skip to content

Commit 2441982

Browse files
vania-poohvania-pooh
committed
Initial commit
0 parents  commit 2441982

File tree

105 files changed

+3978
-0
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

105 files changed

+3978
-0
lines changed

‎.gitignore

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
# IDEA files
2+
.idea
3+
*.iml
4+
5+
# Maven files
6+
target
7+
8+
# Docker compose files
9+
compose/war
10+
11+
# Npm modules
12+
node_modules

‎AUTHORS

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
The following authors have created the source code of "Selenium Grid Router"
2+
published and distributed by YANDEX LLC as the owner:
3+
4+
* Alexander Andryashin <[email protected]>
5+
* Dmitry Baev <[email protected]>
6+
* Artem Eroshenko <[email protected]>
7+
* Innokenty Shuvalov <[email protected]>
8+
* Ivan Krutov <[email protected]>

‎LICENSE

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
(C) YANDEX LLC, 2015
2+
3+
The Source Code called "Selenium Grid Router" available at https://github.com/seleniumkit/gridrouter is subject
4+
to the terms of the Apache License 2.0 (hereinafter referred to as the "License").
5+
The text of the License is the following:
6+
7+
Licensed under the Apache License, Version 2.0 (the "License");
8+
you may not use this file except in compliance with the License.
9+
You may obtain a copy of the License at
10+
11+
http://www.apache.org/licenses/LICENSE-2.0
12+
13+
Unless required by applicable law or agreed to in writing, software
14+
distributed under the License is distributed on an "AS IS" BASIS,
15+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
16+
See the License for the specific language governing permissions and
17+
limitations under the License.

‎README.md

Lines changed: 144 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,144 @@
1+
# Selenium Grid Router
2+
3+
**Selenium Grid Router** is a lightweight server that routes and proxies [Selenium Wedriver](http://www.seleniumhq.org/projects/webdriver/) requests to multiple Selenium hubs.
4+
5+
## What is this for
6+
If you're frequently using Selenium for running your tests in browsers you may notice that a standard [Selenium Grid](https://github.com/SeleniumHQ/selenium/wiki/Grid2) installation has some faults that can prevent you from using it on large scale:
7+
* **Does not support high availability.** Selenium Grid consists of a single entry point **hub** server and multiple **node** processes. Users interact only with hub. That means that if for some reason hub goes down all nodes also become unavailable to user.
8+
* **Does not scale well.** Our experience shows that even when running on high-end hardware Selenium hub is able to handle correctly no more than 20-30 nodes. When more nodes are connected hub very often stops responding.
9+
* **Does not support authentication and authorization.** Standard Selenium grid hub makes all nodes available for everyone.
10+
11+
## How it works
12+
The basic idea is very simple:
13+
14+
1. Define user names and their passwords in a plain text file
15+
2. Distribute a set of running Selenium hubs (aka "hosts") with nodes connected to each one over multiple datacenters
16+
3. For each defined user save hosts to a simple XML configuration file
17+
4. Start multiple instances of Grid Router in different datacenters and load-balance them
18+
5. Work with Grid Router like you do with a regular Selenium hub
19+
20+
## Installation
21+
22+
Currently we maintain only Debian packages. To install on Ubuntu ensure that you have Java 8 installed:
23+
```
24+
# add-apt-repository ppa:webupd8team/java
25+
# apt-get update
26+
# apt-get install oracle-java8-installer
27+
```
28+
Then install Gridrouter itself:
29+
```
30+
# add-apt-repository ppa:yandex-qatools/gridrouter
31+
# apt-get update
32+
# apt-get install yandex-grid-router
33+
# service yandex-grid-router start
34+
```
35+
Configuration files are located in `/etc/grid-router/` directory, log files reside in `/var/log/grid-router/`, binaries are installed to `/usr/share/grid-router`.
36+
37+
## Configuration
38+
Two types of configuration files exist:
39+
* A plain text file with users and passwords (users.properties)
40+
* An XML file with user quota definition (&lt;username&gt;.xml)
41+
42+
### Users list (users.properties)
43+
A typical file looks like this:
44+
```
45+
alice:alicePassword, user
46+
bob:bobPassword, user
47+
```
48+
As you can see passwords are **NOT** encrypted. This is because we consider quotas as a way to easily limit Selenium browsers consumption and not a restrictive tool.
49+
50+
### User quota definition (&lt;username&gt;.xml)
51+
This file has the following format:
52+
```xml
53+
<qa:browsers xmlns:qa="urn:config.gridrouter.qatools.ru">
54+
<browser name="firefox" defaultVersion="33.0">
55+
<version number="33.0">
56+
<region name="us-west">
57+
<host name="my-firefox33-hub-1.example.com" port="4444" count="5"/>
58+
</region>
59+
<region name="us-east">
60+
<host name="my-firefox33-hub-2.example.com" port="4444" count="5"/>
61+
</region>
62+
</version>
63+
<version number="37.0">
64+
<region name="us-west">
65+
<host name="my-firefox37-hub-1.example.com" port="4444" count="3"/>
66+
<host name="my-firefox37-hub-2.example.com" port="4444" count="4"/>
67+
</region>
68+
<region name="us-east">
69+
<host name="my-firefox37-hub-3.example.com" port="4444" count="2"/>
70+
</region>
71+
</version>
72+
</browser>
73+
<browser name="chrome" defaultVersion="42.0">
74+
<version number="42.0">
75+
<region name="us-west">
76+
<host name="my-chrome42-hub-1.example.com" port="4444" count="10"/>
77+
</region>
78+
<region name="us-east">
79+
<host name="my-chrome42-hub-2.example.com" port="4444" count="10"/>
80+
</region>
81+
</version>
82+
</browser>
83+
</qa:browsers>
84+
```
85+
What we basically do in this file - we enumerate hub hosts, ports and counts of browsers available on each hub. We also distribute hosts across regions, i.e. we place hosts from different datacenters in different **&lt;region&gt;** tags. The most important thing is to make sure that browser name and browser version have **exactly** the same value as respective Selenium hub does.
86+
87+
### Authentication
88+
Grid router is using [BASIC HTTP authentication](https://en.wikipedia.org/wiki/Basic_access_authentication). That means that for the majority of test frameworks connection URL would be:
89+
```
90+
http://username:[email protected]:4444/wd/hub
91+
```
92+
However some Javascript test frameworks have their own ways to specify connection URL, user name and password.
93+
94+
### Hub selection logic
95+
When you request a browser by specifying its name and version **Grid Router** does the following:
96+
97+
1. Searches for the browser in user quota XML and returns error if not found
98+
2. Randomly selects a host from all hosts and tries to obtain browser on that host. Our algorithm also considers browser counts specified in XML for each host so that hosts with more browsers get more connections.
99+
3. If browser was obtained - returns it to the user and proxies all requests in this session to the same host
100+
4. In not - selects a new host **from another region** and tries again. This guarantees that when one datacenter goes down in most of cases we'll obtain browser at worst after the second attempt.
101+
5. After trying all hosts returns error if no browser was obtained
102+
103+
### Hub configuration recommendations
104+
Our experience shows that Grid Router works better with a big set of "small" hubs (having no more than 5 connected nodes) than with some "big" hubs. A good idea is to launch small virtual machines (with 1 or 2 virtual CPUs) containing one Selenium hub process 4-5 Selenium node processes that connect to **localhost**. This gives us the following profit:
105+
* Because we have more hubs the probability to successfully obtain browser is greater
106+
* If each virtual machine has only one browser version installed - it's simpler to increase overall count of available browsers
107+
* Hubs with small count of connected nodes perform better
108+
109+
## Development
110+
We're using [Docker](https://www.docker.com/) and [Ansible](http://www.ansible.com/) for integration tests so you need to install them on your Mac or Linux.
111+
112+
### Install Boot2docker (dog-nail for Mac users)
113+
114+
* Install Ansible: `brew install ansible`
115+
* Create an empty inventory file: `touch /usr/local/etc/ansible/hosts`
116+
* Adjust Python settings: `echo 'localhost ansible_python_interpreter=/usr/local/bin/python' >> /usr/local/etc/ansible/hosts`
117+
* Instally Python from [official website](https://www.python.org/ftp/python/2.7.10/python-2.7.10-macosx10.6.pkg)
118+
* Install requests with pip: `pip install requests[security]`
119+
* Install docker-py: `pip install -Iv https://pypi.python.org/packages/source/d/docker-py/docker-py-1.1.0.tar.gz`
120+
* Run boot2docker: `boot2docker up`
121+
* Get Docker VM IP: `boot2docker ip`
122+
* Modify `/etc/hosts`: `<boot2docker_ip> boot2docker`
123+
* Add certificates information to console: `$(boot2docker shellinit)`
124+
* Export correct host name: `export DOCKER_HOST=tcp://boot2docker:2376`
125+
126+
### Running service locally
127+
128+
#### Start
129+
130+
1. Build project: `mvn clean package`
131+
2. Start app: `ansible-playbook testing/start.yml`
132+
3. Check that container is running: `docker ps -a`
133+
134+
#### Run integration tests
135+
136+
```bash]
137+
$ ansible-playbook testing/test.yml
138+
```
139+
140+
#### Stop
141+
142+
```bash
143+
$ ansible-playbook testing/stop.yml
144+
```

0 commit comments

Comments
 (0)