Test offline site (mocked http server)

elgalu · elgalu · commit 393bcbe79734 · 2016-07-12T08:03:09.000+02:00
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -15,6 +15,7 @@ Note image ids also change after scm-source.json has being updated which trigger
  + Add docs/share-host.md and docs/jenkins.md (WIP)
  + Chore: Add `WORKDIR ${NORMAL_USER_HOME}`
  + Chore: Add `docker exec grid cat HUB_PORT #=> 24444`
+ + Chore: Test offline site (mocked http server)
  + Image tag details:
   + Selenium: vTBD_SELENIUM_VERSION (TBD_SELENIUM_REVISION)
   + Chrome stable:  TBD_CHROME_STABLE
diff --git a/Dockerfile b/Dockerfile
@@ -41,7 +41,8 @@ ENV DEBIAN_FRONTEND=noninteractive \
 RUN apt-key adv --recv-keys --keyserver keyserver.ubuntu.com \
       3B4FE6ACC0B21F32 \
       40976EAF437D05B5 \
-      2EA8F35793D8809A
+      2EA8F35793D8809A \
+  && apt-key update -qqy
 
 #========================
 # Miscellaneous packages
@@ -893,6 +894,7 @@ ADD **/etc/supervisor/conf.d/* /etc/supervisor/conf.d/
 #==============================================
 # See: SeleniumHQ/docker-selenium/issues/14
 RUN apt-get update -qqy \
+  && apt-key update -qqy \
   && apt-get -qqy install \
     haveged rng-tools \
   && service haveged start \
@@ -975,7 +977,7 @@ ENV FIREFOX_VERSION="${FF_VER}" \
   # How often in ms the node will try to register itself again.
   # Allow to restart the hub without having to restart the nodes.
   #  (node) in ms. Selenium default: 5000
-  SELENIUM_NODE_REGISTER_CYCLE="0" \
+  SELENIUM_NODE_REGISTER_CYCLE="5000" \
   # To taggle issue #58 see https://goo.gl/fz6RTu
   CHROME_ARGS="--no-sandbox" \
   # e.g. CHROME_ARGS="--no-sandbox --ignore-certificate-errors" \
diff --git a/README.md b/README.md
@@ -100,11 +100,11 @@ See [docker-compose](./docs/docker-compose.md)
 See [jenkins](./docs/jenkins.md)
 
 ### Parallel
-This image is designed to run one test on each docker container but if you still want to run multiple tests in parallel you can still do, there are some ways to do this:
+This image is designed to run one test on each docker container but if you still want to run multiple tests in parallel you can still do so, there are some ways to do this:
 
-1. The recommended way is via [docker-compose](./docs/docker-compose.md)
+1. The recommended way is via [docker-compose](./docs/docker-compose.md) and you should replace `adwords_mock` with your web service under test within the [docker-compose.yml][] file.
 
-        SELENIUM_HUB_PORT=4444 docker-compose -p selenium scale hub=1 chrome=3 firefox=3
+        SELENIUM_HUB_PORT=4444 docker-compose -p selenium scale adwords_mock=1 hub=1 chrome=3 firefox=3
 
 1. The _(not recommended)_ way is by increasing `MAX_INSTANCES` and `MAX_SESSIONS` which now [defaults](https://github.com/elgalu/docker-selenium/blob/2.53.1a/Dockerfile#L967) to 1.
 
@@ -499,3 +499,4 @@ Powered by Supervisor, the container leaves many logs;
 [sauce]: https://saucelabs.com/selenium/selenium-grid
 [BrowserStack]: https://www.browserstack.com/automate
 [xvfb-electron]: http://electron.atom.io/docs/tutorial/testing-on-headless-ci
+[docker-compose.yml]: ./docker-compose.yml
diff --git a/docker-compose-host.yml b/docker-compose-host.yml
@@ -11,11 +11,10 @@ services:
       - GRID=true
       - CHROME=false
       - FIREFOX=false
+      - MOCK_SERVER_PORT=33001
 
   chrome:
     image: elgalu/selenium
-    depends_on:
-      - hub
     network_mode: host
     shm_size: 1g
     environment:
@@ -25,11 +24,10 @@ services:
       - GRID=false
       - CHROME=true
       - FIREFOX=false
+      - MOCK_SERVER_PORT=33001
 
   firefox:
     image: elgalu/selenium
-    depends_on:
-      - hub
     network_mode: host
     environment:
       - SELENIUM_HUB_PORT
@@ -38,6 +36,13 @@ services:
       - GRID=false
       - CHROME=false
       - FIREFOX=true
+      - MOCK_SERVER_PORT=33001
+
+  adwords_mock:
+    image: elgalu/google_adwords_mock
+    network_mode: host
+    environment:
+      - MOCK_SERVER_PORT=33001
 
 #-- Notes --#
 # shm_size: 1g
diff --git a/docker-compose.yml b/docker-compose.yml
@@ -12,6 +12,7 @@ services:
       - GRID=true
       - CHROME=false
       - FIREFOX=false
+      - MOCK_SERVER_PORT=33001
 
   chrome:
     image: elgalu/selenium
@@ -24,6 +25,7 @@ services:
       - GRID=false
       - CHROME=true
       - FIREFOX=false
+      - MOCK_SERVER_PORT=33001
 
   firefox:
     image: elgalu/selenium
@@ -35,6 +37,14 @@ services:
       - GRID=false
       - CHROME=false
       - FIREFOX=true
+      - MOCK_SERVER_PORT=33001
+
+  adwords_mock:
+    image: elgalu/google_adwords_mock
+    network_mode: "service:hub"
+    environment:
+      - MOCK_SERVER_PORT=33001
+
 
 #-- Notes --#
 # shm_size: 1g
diff --git a/docs/docker-compose.md b/docs/docker-compose.md
@@ -13,6 +13,7 @@ Either clone this repository or download the file [docker-compose.yml][] using `
 
 ### Run
 Start it with `docker-compose up` then **scale** it:
+You should replace `adwords_mock` with your web service under test within the [docker-compose.yml][] file.
 
     export SELENIUM_HUB_PORT=4444 NODES=3
     docker-compose -p selenium up -d
@@ -29,8 +30,8 @@ Wait until the grid starts properly before starting the tests _(Optional but rec
 ### Test
 You can now run your tests by using the `--seleniumUrl="http://localhost:4444/wd/hub"`.
 
-However if your web application under test is running in localhost, e.g. `--appHost=localhost`
-you should instead either dockerize your application and add it to the [docker-compose.yml][] file as another [service](https://docs.docker.com/compose/compose-file/#/service-configuration-reference) or somehow replace `--appHost=localhost` with `--appHost=d.host.loc.dev` in the config file of your testing framework. The string `d.host.loc.dev` is a place holder inside the docker container that points to the IP of your localhost.
+However if your web application under test is running in localhost, e.g. `--appHost=localhost` or is publicly not accessible
+you should instead either dockerize your application as shown in the example [adwords_mock](https://github.com/elgalu/google_adwords_mock) within the [docker-compose.yml][]. Its there shown as another [service](https://docs.docker.com/compose/compose-file/#/service-configuration-reference). Another option (if you don't want to dockerize your app yet) is to somehow replace `--appHost=localhost` with `--appHost=d.host.loc.dev` in the config file of your testing framework. The string `d.host.loc.dev` is a place holder inside the docker container that points to the IP of your localhost.
 
 ### Cleanup
 Once your tests are done you can clean up:
diff --git a/docs/share-host.md b/docs/share-host.md
@@ -6,17 +6,18 @@ For using it non-Linux machines or, in general, by not sharing the host (localho
 For requirements check [README#requisites](../README.md#requisites)
 
 ## Usage
-Either clone this repository or download the file [docker-compose-host.yml](../docker-compose-host.yml) using `wget`
+Either clone this repository or download the file [docker-compose-host.yml][] using `wget`
 
     wget -nv "https://raw.githubusercontent.com/elgalu/docker-selenium/master/docker-compose-host.yml"
     mv -f docker-compose-host.yml docker-compose.yml
     docker-compose -p selenium down #ensure is not already running
 
 ### Run
 Either start with `docker-compose ... scale` as shown in below example or you can also use `docker-compose up` and scale after in a second command.
+You should replace `adwords_mock` with your web service under test within the [docker-compose-host.yml][] file..
 
     export SELENIUM_HUB_PORT=4444 NODES=3
-    docker-compose -p selenium scale hub=1 chrome=${NODES} firefox=${NODES}
+    docker-compose -p selenium scale adwords_mock=1 hub=1 chrome=${NODES} firefox=${NODES}
 
 Wait until the grid starts properly before starting the tests _(Optional but recommended)_
 
@@ -36,3 +37,5 @@ Once your tests are done you can clean up:
     docker-compose -p selenium down
 
 The `down` compose command stops and remove containers, networks, volumes, and images created by `up` or `scale`
+
+[docker-compose-host.yml]: ../docker-compose-host.yml
diff --git a/misc/misc.md b/misc/misc.md
@@ -3,7 +3,7 @@ Miscellaneous internal notes, do not read!
 ## Build
 
     time (docker build -t elgalu/selenium . ;echo $?;beep)
-    docker run --rm -ti -m 3000M --name=local -p=4444:24444 -p=5900:25900 -e VIDEO=true -v /dev/shm:/dev/shm elgalu/selenium
+    docker run --rm -ti -m 3000M --name=local -e SELENIUM_HUB_PORT=4444 -p=4444:4444 -p=5900:25900 -e VIDEO=true -e VNC_PASSWORD=hola -v /dev/shm:/dev/shm elgalu/selenium
 
 ### Wait
 Wait and get versions
diff --git a/test/after_script b/test/after_script
@@ -7,6 +7,5 @@ set -e
 docker exec grid stop || true
 docker stop grid || true
 docker rm grid || true
-
-# fast wait
-# docker rm -vf grid || true
+# quick destroy
+docker rm -vf grid_adwords_mock || true
diff --git a/test/compose-test.sh b/test/compose-test.sh
@@ -33,7 +33,7 @@ if [ "${DO_COMPOSE_UP}" = "true" ]; then
 fi
 
 # Compose scale!
-docker-compose -f ${COMPOSE_FILE} -p selenium scale hub=1 chrome=${NUM_NODES} firefox=${NUM_NODES}
+docker-compose -f ${COMPOSE_FILE} -p selenium scale adwords_mock=1 hub=1 chrome=${NUM_NODES} firefox=${NUM_NODES}
 
 # FIXME: We still need to wait a bit because the nodes registration is not
 #        being waited on wait_all_done script :(
diff --git a/test/install_grid b/test/install_grid
@@ -8,8 +8,13 @@ set -xe
 
 # Ensure clean env
 docker rm -vf grid || true
+docker rm -vf grid_adwords_mock || true
 
 docker run --name=grid -d -e VIDEO=true -v /dev/shm:/dev/shm selenium
+
+# We need a mock service for testing
+docker run -d --name=grid_adwords_mock --net=container:grid elgalu/google_adwords_mock
+
 docker exec grid wait_all_done 40s
 docker exec grid versions
 docker exec grid errors || true
diff --git a/test/python_test.py b/test/python_test.py
@@ -40,7 +40,7 @@
 
 from retrying import retry
 
-@retry(stop_max_attempt_number=5, stop_max_delay=180100, wait_fixed=200)
+@retry(stop_max_attempt_number=12, stop_max_delay=180100, wait_fixed=300)
 def webdriver_connect():
     # http://selenium-python.readthedocs.org/en/latest/getting-started.html#using-selenium-with-remote-webdriver
     return webdriver.Remote(command_executor=myselenium_hub_url, desired_capabilities=caps)
@@ -79,32 +79,47 @@ def webdriver_connect():
 driver.set_window_size(width, height)
 
 # Test: https://code.google.com/p/chromium/issues/detail?id=519952
-pageurl = "http://www.google.com/adwords"
-# pageurl = "http://d.host.loc.dev:8088/adwords"
-print ("Opening page %s" % pageurl)
-driver.get(pageurl)
+# e.g. pageurl = "http://localhost:33001/adwords"
+# e.g. pageurl = "http://www.google.com:80/adwords"
+# e.g. pageurl = "https://www.google.com:443/adwords"
+# e.g. pageurl = "http://d.host.loc.dev:33001/adwords"
+page_port = os.environ.get('MOCK_SERVER_PORT','33001')
+page_host = os.environ.get('MOCK_SERVER_HOST','localhost')
+pageurl = ("http://%s:%s/adwords" % (page_host, page_port))
+
+@retry(stop_max_attempt_number=7, stop_max_delay=30100, wait_fixed=300)
+def open_web_page():
+    print ("Opening page %s" % pageurl)
+    driver.get(pageurl)
+    time.sleep(msleep)
+    print ("Current title: %s" % driver.title)
+    print ("Asserting 'Google Adwords' in driver.title")
+    assert "Google AdWords | Pay-per-Click-Onlinewerbung auf Google (PPC)" in driver.title
+
+open_web_page()
+
+print ("Click link 'Kosten'")
+link = driver.find_element_by_link_text('Kosten')
+link.click()
+driver.maximize_window()
+time.sleep(msleep)
+
+@retry(stop_max_attempt_number=7, stop_max_delay=20100, wait_fixed=300)
+def open_costs_page():
+    print ("Current title: %s" % driver.title)
+    print ("Asserting 'Kosten' in driver.title")
+    assert "Kosten von Google AdWords | Google AdWords" in driver.title
+
+open_costs_page()
+
+print ("Go back to home page")
+link = driver.find_element_by_link_text('Übersicht')
+link.click()
+time.sleep(msleep)
 print ("Current title: %s" % driver.title)
-print ("Asserting 'Google Adwords' in driver.title")
-assert "Google AdWords" in driver.title
-# assert "Google AdWords | Pay-per-Click-Onlinewerbung auf Google (PPC)" in driver.title
-
-# print ("Click link 'Kosten'")
-# link = driver.find_element_by_link_text('Kosten')
-# link.click()
-# driver.maximize_window()
-# time.sleep(msleep)
-# print ("Current title: %s" % driver.title)
-# print ("Asserting 'Kosten' in driver.title")
-# assert "Kosten von Google AdWords | Google AdWords" in driver.title
-
-# print ("Go back to home page")
-# link = driver.find_element_by_link_text('Übersicht')
-# link.click()
-# time.sleep(msleep)
-# print ("Current title: %s" % driver.title)
-# print ("Asserting 'Google (PPC)' in driver.title")
-# assert "Google AdWords | Pay-per-Click-Onlinewerbung auf Google (PPC)" in driver.title
-# time.sleep(msleep)
+print ("Asserting 'Google (PPC)' in driver.title")
+assert "Google AdWords | Pay-per-Click-Onlinewerbung auf Google (PPC)" in driver.title
+time.sleep(msleep)
 
 print ("Close driver and clean up")
 driver.close()
diff --git a/test/script_scenario_node_dies b/test/script_scenario_node_dies
@@ -37,6 +37,7 @@ echo "#================================================="
 
 # Ensure clean env
 docker rm -vf mynodes || true
+docker rm -vf grid_adwords_mock || true
 
 # Create container for Chrome
 docker run --name=mynodes -d \
@@ -45,6 +46,9 @@ docker run --name=mynodes -d \
   -v /dev/shm:/dev/shm \
   selenium
 
+# We need a mock service for testing
+docker run -d --name=grid_adwords_mock --net=container:mynodes elgalu/google_adwords_mock
+
 docker exec mynodes wait_all_done 40s
 docker exec mynodes errors || true
 docker logs mynodes
@@ -73,13 +77,17 @@ echo "#================================================="
 
 # Ensure clean env
 docker rm -vf mynodes || true
+docker rm -vf grid_adwords_mock || true
 
 # Create container for Firefox
 docker run --name=mynodes -d \
   -e CHROME=false \
   -e SELENIUM_NODE_FF_PORT -p ${SELENIUM_NODE_FF_PORT}:${SELENIUM_NODE_FF_PORT} \
   selenium
 
+# We need a mock service for testing
+docker run -d --name=grid_adwords_mock --net=container:mynodes elgalu/google_adwords_mock
+
 docker exec mynodes wait_all_done 40s
 docker exec mynodes errors || true
 docker logs mynodes
diff --git a/test/script_scenario_restart b/test/script_scenario_restart
@@ -16,6 +16,7 @@ echo "#======================================"
 echo "# Scenario 2a [restart]: Normal restart"
 echo "#======================================"
 docker restart grid #normal restart
+docker restart grid_adwords_mock
 sleep ${SLEEP_RACE_COND_SECS}
 docker exec grid wait_all_done ${WAIT_ALL_DONE}
 docker logs grid
@@ -26,6 +27,7 @@ echo "#========================================================================"
 echo "# Scenario 2b [restart]: Forced quick container restart + run tests again"
 echo "#========================================================================"
 docker restart --time=0 grid #forced quick restart
+docker restart --time=0 grid_adwords_mock
 sleep ${SLEEP_RACE_COND_SECS}
 docker exec grid wait_all_done ${WAIT_ALL_DONE}
 docker logs grid
