Improve deploy check when app repeatedly crashes

The ip address changes each time the app restarts,  which caused problems. To improve it:
- The ip address is retrieved right before it is used
- If there is no ip address, it skips the curl command
- Add a 10 second timeout to the curl command. If the ip address is no longer used by docker, it will instead try to load the page from a remote server with the ip address, which can take a while.
- If the app is restarting during 10 checks, it will revert the app without waiting the full time

Author: zodern

CHANGELOG.md

@@ -25,13 +25,15 @@ The `mup status` command gives an overview of what is running on the servers and
 - `mup init` will create a `.deploy` folder when run in the same folder as a Meteor app
 - When mup can find a meteor app near to where `mup init` is run, the default config's `app.path` will be the path to that app
 - When a deploy fails, the last 200 instead of 100 lines of the app's logs are shown
-- More of the logs are shown when a command fails
+- More of the output is shown when a command fails
 - When copying a file fails with the error `No such file`, it will tell the user to run `mup setup` to fix it
 - `reconfig` hooks will now run during `mup deploy`
 - `--show` is no longer needed to show the config when `mup validate --scrub` is run
 - Add section to readme about Meteor compatibility
 - Initial work has been done to support Docker Swarm
 - When there is only one server, `mup ssh` will not require the name of a server
+- If the app's docker container is restarting during 10 checks, the Deployment Verifier will revert the app without waiting the full time in `deployCheckWaitTime`
+- Added 10 second timeout to the curl command in the Deployment Verifier
 - Fix retry logic for the copy file task
 - Fix running Prepare Bundle when image already has a `/built_app` folder
 - Fix alignment of list of servers when running `mup ssh` without specifying a server

src/plugins/meteor/assets/meteor-deploy-check.sh

@@ -36,21 +36,42 @@ revert_app (){
 }
 
 elaspsed=0
+noIPCount=0
+MAX_NO_IP_COUNT=10
 while [[ true ]]; do
+  if [ "$elaspsed" "==" "$DEPLOY_CHECK_WAIT_TIME" ]; then
+    revert_app
+    exit 1
+  fi
+
   sleep 1
   elaspsed=$((elaspsed+1))
 
+  # If the container restarted, the ip address would have changed
+  # Get the current ip address right before it is used
+  CONTAINER_IP=$(docker inspect $APPNAME --format "{{.NetworkSettings.IPAddress}}")
+
+  if [[ -z $CONTAINER_IP ]]; then
+    noIPCount=$((noIPCount+1))
+
+    if [ "$noIPCount" "==" "$MAX_NO_IP_COUNT" ]; then
+      echo "Too much time spent restarting." 1>&2
+      revert_app
+      exit 1
+    fi
+
+    continue
+  fi
+
+  DEPLOY_CHECK_URL=$CONTAINER_IP<%= `:${deployCheckPort}` %>
+
   # Since this failing causes the app to rollback, it should only
   # fail because of a problem with the app, not from problems with the config.
   #
   # --insecure Without this, it would sometimes fail when ssl is set up
   curl \
+    --max-time 10 \
     --insecure \
     $DEPLOY_CHECK_URL \
     && exit 0
-
-  if [ "$elaspsed" "==" "$DEPLOY_CHECK_WAIT_TIME" ]; then
-    revert_app
-    exit 1
-  fi
 done