cherry-pick 3a965a0 from release-1.6

Wait until the webhook is "actually" ready before running the smoke
tests.

Signed-off-by: Maël Valais <[email protected]>

Author: maelvls

smoke-test.Dockerfile

@@ -9,11 +9,9 @@
 # Dockerfile: https://github.com/GoogleCloudPlatform/marketplace-testrunner/blob/master/Dockerfile
 FROM python:alpine
 
-RUN apk add curl patch
+RUN apk add curl patch bash
 RUN pip3 install cram
-RUN curl -L https://github.com/stern/stern/releases/download/v1.19.0/stern_1.19.0_linux_amd64.tar.gz | tar xz -C /tmp \
-    && mv /tmp/stern_1.19.0_linux_amd64/stern /usr/local/bin \
-    && curl -L https://storage.googleapis.com/kubernetes-release/release/v1.19.6/bin/linux/amd64/kubectl --output-dir /usr/local/bin -O \
+RUN curl -L https://storage.googleapis.com/kubernetes-release/release/v1.19.6/bin/linux/amd64/kubectl --output-dir /usr/local/bin -O \
     && chmod 755 /usr/local/bin/kubectl
 
 WORKDIR /opt
@@ -24,4 +22,4 @@ ENV NAMESPACE=test
 # We don't use "kubectl logs" because we need to know to which container each
 # logs comes from. Also, stern follows logs and never returns, so we do the
 # pause trick.
-CMD ["sh", "-c", "cram smoke-test.t || (stern -A -l app.kubernetes.io/name=jetstack-secure-gcm & (sleep 5; exit 1))"]
+CMD ["sh", "-c", "cram smoke-test.t"]

smoke-test.t

@@ -7,7 +7,22 @@ test suite. This image is used in a Job in data-test. mpdev verify detects the
 presence of this Job thanks to the annotation
  "marketplace.cloud.google.com/verification: test"
 
-First, let us create a self-signed Issuer and a Certificate. This is a good way
+Let's make sure the cert-manager API is actually ready. We do 2>/dev/null due to
+the fact that this command outputs an unknown number of lines, which is
+something I don't know how to express in cram syntax.
+
+  $ timeout 5m bash -c "until kubectl apply --dry-run=server -f- <<<$'kind: Issuer\napiVersion: cert-manager.io/v1\nmetadata:\n  name: a\nspec:\n  selfSigned: {}' 2>/dev/null >&2; do sleep 10s; done"
+  $ kubectl apply --dry-run=server -f- <<EOF
+  > kind: Issuer
+  > apiVersion: cert-manager.io/v1
+  > metadata:
+  >   name: a
+  > spec:
+  >   selfSigned: {}
+  > EOF
+  issuer.cert-manager.io/a created (server dry run)
+
+Let us create a self-signed Issuer and a Certificate. This is a good way
 to spot webhook misconfigurations.
 
   $ kubectl apply -n ${NAMESPACE} -f - <<EOF