Skip to content

Travis TEST_DEPLOYED_IMAGE tests wrong image on pull requests #109

@hannahilea

Description

@hannahilea

Right now, we always run the Travis TEST_DEPLOYED_IMAGE stage. [1] The one job at this stage pulls an image from dockerhub and runs the regression tests on it.

Because we do not always push images to docker hub---in fact, we only push them on merges to master, not on pull requests! [2] --- the image under test is NOT THE PR'S IMAGE for a pull request, because that image has not been pushed to docker hub. Instead, it is the pre-existing pushed image (i.e., the most recent master branch build) with the same tag name.

This seems...incorrect? I think we should either be

A couple fix approaches:

  1. Only run the TEST_DEPLOYED_IMAGE stage on merges to master, NOT pull requests
  2. Always push the built image to docker hub, under a name specific to the branch...then run TEST_DEPLOYED_IMAGE on that branch-specific name

I think the quick fix is to do option 1 (so that we're not inadvertently testing the wrong thing!) and then leave this issue open until we've decided if we also want to do 2.

[1]

if: branch = master
, where

branch (the current branch name; for pull requests: the base branch name) ref

[2]

- if test "${TRAVIS_PULL_REQUEST}" = "false"; then export DEPLOY_ME="true"; else export DEPLOY_ME="false"; fi

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions