Codecov failure rate

[davida72]

I've been looking at the unit tests which run every night and notice that the failure rate is high for a lot of councils. Is that because the script is broken?

I ran the unit tests myself, but didn't have selenium set up at localhost therefore many failed. Will I have contributed to the stats at Codecov?

[robbrad]

I would highly recommend using the devcontainer as it give you everything you need to do development work (even selenium) out of the box.

There are two types of testing used in this project

1/ Unit tests test the static code - eg functions like check postcode, the wrapper functions, ability to create web driver (generic stuff)

2/ integration tests - these test each (changed) council on pull request and a nightly run of each council on the master branch(do check the result branch that's related in code cov)

Anything you do locally won't upload to code cov.

The reasons for the test success rate could be a number of things - council outages / code issues etc.

There are a lot more with no failure than there are failures.

The point about nightly is to keep track of councils where they waste their money implementing new websites that then need an integration change.

[davida72]

Thank you.

Apologies if that came across as a complaint. I love this project and love the fact that it has tests set up. I hope I'll be able to make some fixes and add some councils, but I only found the 'contributing' page yesterday so I'm still getting properly set up.

I'll try using the devcontainer.

[robbrad]

Not at all!

The codecov UI doesn't really display the results as well as the allure report did from a historical perspective.

Where this helps me is due to the projects nature, lots of different people contribute and this standardisation of testing is a great way to protect the way the project performs