-
Notifications
You must be signed in to change notification settings - Fork 12
Description
Right now, we always run the Travis TEST_DEPLOYED_IMAGE stage. [1] The one job at this stage pulls an image from dockerhub and runs the regression tests on it.
Because we do not always push images to docker hub---in fact, we only push them on merges to master, not on pull requests! [2] --- the image under test is NOT THE PR'S IMAGE for a pull request, because that image has not been pushed to docker hub. Instead, it is the pre-existing pushed image (i.e., the most recent master branch build) with the same tag name.
This seems...incorrect? I think we should either be
A couple fix approaches:
- Only run the
TEST_DEPLOYED_IMAGEstage on merges to master, NOT pull requests - Always push the built image to docker hub, under a name specific to the branch...then run
TEST_DEPLOYED_IMAGEon that branch-specific name
I think the quick fix is to do option 1 (so that we're not inadvertently testing the wrong thing!) and then leave this issue open until we've decided if we also want to do 2.
[1]
Line 96 in 25d3f74
| if: branch = master |
branch (the current branch name; for pull requests: the base branch name) ref
[2]
Line 68 in 25d3f74
| - if test "${TRAVIS_PULL_REQUEST}" = "false"; then export DEPLOY_ME="true"; else export DEPLOY_ME="false"; fi |