Integrate regression testing

[pun-ky]

Using: https://github.com/BBC-News/wraith
Due to requirement about having ImageMagick installed on machine, it should be deployed on Docker via GAP environment plugin:
http://bbc-news.github.io/wraith/os-install.html
https://hub.docker.com/r/bbcnews/wraith/dockerfile/

Alternatively consider using AET https://github.com/Skejven/aet-docker , but seems that it is just more heavy for local dev machine (but better for production :) / but it is not the case here)

[mmajchrzak]

I would opt for using AET for a number of reasons:

• more comprehensive than Wraith, e.g. comparing various breakpoints, js+http errors, advanced control on image comparison
• reusing tests across local and shared envs. If you go for Wraith on local and AET on shared envs, you'll end up with test duplication which is a really bad idea - nobody would like to maintain such a setup

I appreciate the fact that AET may be heavy on local machine. In this case I'd cooperate with AET team on how we can make the setup lighter rather than investing the effort in integrating Wraith.

[pun-ky]

@Skejven how about creating compact / portable / lightweight version of AET? ;)

[malaskowski]

Hi,

compact / portable / lightweight version of AET

This idea was in my mind for some time...
The whole complication is that the most "heavy" in AET setup are browsers and database.
While I can imagine getting rid of the latter (with some e.g. in-memory storage), then AET would rather be useless without access to the browser...
What comes to my mind when we talk about AET-lite is using the host browser. This has some disadvantages though (like handling browser auto-update etc.), which might not be a case in the dev env.

[mmajchrzak]

So if we make the database aspect of AET lighter, then AET should be comparable to Wraith, right? @pun-ky have you validated how heavy the Wraith is? It also uses headless browsers (selenium+Chromedriver)...

[malaskowski]

Well, it looks that Wraith is just comparing images. AET has quite a wider range of possibilities :)

[pun-ky]

Just looked for alternatives. No. I just want to check screenshots, really (the simpler solution the better, the plan is to check it for each build, not ocassionaly so the time is crucial). All other stuff probably could be handled by webhint and lighthouse.

[pun-ky]

Another option could be to try with https://github.com/palmerhq/cypress-image-snapshot as of there are already plans to check also cypress.io in case of functional tests