Smoke tests for the Google CAS issuer

.gitattributes

@@ -0,0 +1,6 @@
+# The .t files are cram tests. The GitHub syntax highlighter [1] does not know
+# about the cram format and thinks they are Perl files. So we disabled the
+# GitHub highlighter altogether for these files.
+
+*.t text linguist-language=Text
+

.gitignore

@@ -1,2 +1,3 @@
 ca.crt
 ca.key
+*.t.err

chart/jetstack-secure-gcm/templates/application.yaml

@@ -44,7 +44,6 @@ spec:
         url: https://cert-manager.io/docs/usage/
       - description: Getting Started with the Jetstack Secure Platform
         url: https://platform.jetstack.io/docs
-    info: []
     notes: |-
       ### Create your first certificate
 

cloudbuild.yaml

@@ -144,7 +144,7 @@ steps:
     name: gcr.io/cloud-builders/gcloud
     entrypoint: bash
     args:
-      - -exc
+      - -euc
       - |
         while : ; do
           kubectl get ns -oname 2>/dev/null | grep apptest- && break
@@ -158,6 +158,7 @@ steps:
         done
         pod=$(kubectl -n "$ns" get pods -oname | grep "apptest-.*-deployer" | cut -d/ -f2)
 
+        set -x
         kubectl wait -n "$ns" --for=condition=ready --timeout=5m pod $pod
         kubectl logs -n "$ns" $pod -f --tail=-1
     waitFor:
@@ -168,7 +169,7 @@ steps:
     name: gcr.io/cloud-builders/gcloud
     entrypoint: bash
     args:
-      - -exc
+      - -euc
       - |
         while : ; do
           kubectl get ns -oname 2>/dev/null | grep apptest- && break
@@ -181,6 +182,7 @@ steps:
           sleep 10
           done
 
+        set -x
         kubectl wait -n "$ns" --for=condition=ready --timeout=5m pod smoke-test-pod
         kubectl logs -n "$ns" smoke-test-pod -f --tail=-1
     waitFor:

docs/TESTING-DEPLOYER.md

@@ -10,17 +10,51 @@
 
 **Contents:**
 
+- [Cutting a new release](#cutting-a-new-release)
 - [Pricing mechanism](#pricing-mechanism)
 - [Creating and testing the deployer image](#creating-and-testing-the-deployer-image)
 - [mpdev install on your own cluster](#mpdev-install-on-your-own-cluster)
-- [Cutting a new release](#cutting-a-new-release)
 - [Testing the application without having access to the Billing API](#testing-the-application-without-having-access-to-the-billing-api)
 - [How the Application object "wrangles" its components](#how-the-application-object-wrangles-its-components)
 - [Installing and manually testing the deployer image](#installing-and-manually-testing-the-deployer-image)
 - [Testing and releasing the deployer using Google Cloud Build](#testing-and-releasing-the-deployer-using-google-cloud-build)
   - [Debugging deployer and smoke-tests when run in Cloud Build](#debugging-deployer-and-smoke-tests-when-run-in-cloud-build)
 - [Updating the upstream cert-manager chart version](#updating-the-upstream-cert-manager-chart-version)
 
+## Cutting a new release
+
+Since the process is manual and evolves from release to release, we document all
+the steps that were taken in each release directly on the GitHub Release itself
+in a `<details>` block that looks like this:
+
+> ▶ 📦 Recording of the manual steps of the release process
+
+Imagining that you want to release `1.1.0-gcm.5`, the steps are:
+
+1. Copy the `<details>` block from the previous release [1.1.0-gcm.4](https://github.com/jetstack/jetstack-secure-gcm/releases/tag/1.1.0-gcm.4)
+2. In an editor, change the references to `1.1.0-gcm.4`.
+3. Follow the steps and tick the checkboxes.
+4. After the `1.1.0-gcm.5` is pushed to GitHub, create a GitHub Release for that
+   tag and paste the content into the `<details>` block into the GitHub Release
+   you just created (see `PASTE HERE` below). The GitHub Release description
+   should look like this:
+
+   ```md
+   ## Changelog
+
+   <!-- TODO -->
+
+   ## Notes
+
+   <details>
+
+   <summary>📦 Recording of the manual steps of the release process</summary>
+
+   <!-- PASTE HERE -->
+
+   </details>
+   ```
+
 ## Pricing mechanism
 
 Each cluster is priced at $50 a month, billed hourly ($0.07/hour). The way the
@@ -32,10 +66,10 @@ of `1` to the `time` value. The unit for `time` is something we have configured
 in the [pricing
 panel](https://console.cloud.google.com/partner/editor/jetstack-public/jetstack-secure-for-cert-manager?project=jetstack-public&authuser=4&form=saasK8sPricingPanel).
 
-| Field          | Value  |
-| -------------- | ------ |
-| ID             | `time` |
-| Unit           | `h`    |
+| Field | Value  |
+| ----- | ------ |
+| ID    | `time` |
+| Unit  | `h`    |
 
 Note that the cert-manager deployment should always be run with replicas=1.
 High-availability (replicas > 1) is not supported yet, and the application will
@@ -298,22 +332,6 @@ see everything green:
 
 <img src="https://user-images.githubusercontent.com/2195781/110795922-a96acd00-8277-11eb-959e-bf7ea51ae992.png" width="500" alt="The application page for test-1 shows that all the deployments are green. This screenshot is stored in this issue: https://github.com/jetstack/jetstack-secure-gcm/issues/21">
 
-## Cutting a new release
-
-Since the process is manual and evolves from release to release, we document all
-the steps taken in each release directly on the GitHub Release itself in a
-`<details>` block that looks like this:
-
-> ▶ 📦 Recording of the manual steps of the release process
-
-For example, when releasing `1.1.0-gcm.5`, the steps were:
-
-1. Copy the `<details>` block from the previous release [1.1.0-gcm.4](https://github.com/jetstack/jetstack-secure-gcm/releases/tag/1.1.0-gcm.4)
-2. In an editor, change the references to `1.1.0-gcm.4`.
-3. Follow the steps and tick the checkboxes.
-4. After the `1.1.0-gcm.5` is pushed to GitHub, create a GitHub Release for that
-   tag and paste the content the `<details>` block to the GitHub Release.
-
 ## Testing the application without having access to the Billing API
 
 Jetstack members do not have access to the Billing API. In order to test
@@ -552,7 +570,7 @@ the example [suite.yaml](https://github.com/GoogleCloudPlatform/marketplace-test
 
 ```yaml
 actions:
-  - name: { { .Env.TEST_NAME } }
+  - name: "{{ .Env.TEST_NAME }}"
     httpTest:
       url: http://{{ .Var.MainVmIp }}:9012
       expect:

schema.yaml

@@ -14,7 +14,7 @@ x-google-marketplace:
   #
   # Important: it must have the same value as the version in the
   # Application manifest.
-  publishedVersion: 1.3.1-gcm.0
+  publishedVersion: 1.4.0-gcm.0
   publishedVersionMetadata:
     releaseNote: >-
       Initial release.
@@ -107,6 +107,12 @@ properties:
       serviceAccount:
         description: Service account used by cert-manager Controller Deployment
         roles:
+          - type: Role
+            rulesType: CUSTOM
+            rules:
+              - apiGroups: [""]
+                resources: ["events"]
+                verbs: ["create", "patch"]
           - type: ClusterRole
             rulesType: CUSTOM
             rules:
@@ -120,9 +126,6 @@ properties:
               - apiGroups: [""]
                 resources: ["secrets"]
                 verbs: ["get", "list", "watch", "create", "update", "delete"]
-              - apiGroups: [""]
-                resources: ["events"]
-                verbs: ["create", "patch"]
               # ClusterIssuer controller role
               - apiGroups: ["cert-manager.io"]
                 resources: ["clusterissuers", "clusterissuers/status"]
@@ -133,9 +136,6 @@ properties:
               - apiGroups: [""]
                 resources: ["secrets"]
                 verbs: ["get", "list", "watch", "create", "update", "delete"]
-              - apiGroups: [""]
-                resources: ["events"]
-                verbs: ["create", "patch"]
               # Certificates controller role
               - apiGroups: ["cert-manager.io"]
                 resources: ["certificates", "certificates/status", "certificaterequests", "certificaterequests/status"]
@@ -152,9 +152,6 @@ properties:
               - apiGroups: [""]
                 resources: ["secrets"]
                 verbs: ["get", "list", "watch", "create", "update", "delete"]
-              - apiGroups: [""]
-                resources: ["events"]
-                verbs: ["create", "patch"]
               # Orders controller role
               - apiGroups: ["acme.cert-manager.io"]
                 resources: ["orders", "orders/status"]
@@ -174,9 +171,6 @@ properties:
               - apiGroups: [""]
                 resources: ["secrets"]
                 verbs: ["get", "list", "watch"]
-              - apiGroups: [""]
-                resources: ["events"]
-                verbs: ["create", "patch"]
               # Challenges controller role
               - apiGroups: ["acme.cert-manager.io"]
                 resources: ["challenges", "challenges/status"]
@@ -190,9 +184,6 @@ properties:
               - apiGroups: [""]
                 resources: ["secrets"]
                 verbs: ["get", "list", "watch"]
-              - apiGroups: [""]
-                resources: ["events"]
-                verbs: ["create", "patch"]
               - apiGroups: [""]
                 resources: ["pods", "services"]
                 verbs: ["get", "list", "watch", "create", "delete"]
@@ -226,12 +217,7 @@ properties:
               - apiGroups: ["networking.k8s.io"]
                 resources: ["ingresses"]
                 verbs: ["get", "list", "watch"]
-              - apiGroups: [""]
-                resources: ["events"]
-                verbs: ["create", "patch"]
-              - apiGroups: [""]
-                resources: ["configmaps"]
-                verbs: ["get", "create", "update", "patch"]
+
               - apiGroups: ["cert-manager.io"]
                 resources: ["signers"]
                 verbs: ["approve"]
@@ -242,6 +228,43 @@ properties:
                   - googlecasclusterissuers.cas-issuer.jetstack.io/*
                   - googlecasissuers.cas-issuer.jetstack.io/*
 
+              # Added in https://github.com/jetstack/cert-manager/pull/4064
+              - apiGroups: ["certificates.k8s.io"]
+                resources: ["certificatesigningrequests"]
+                verbs: ["get", "list", "watch", "update"]
+              - apiGroups: ["certificates.k8s.io"]
+                resources: ["certificatesigningrequests/status"]
+                verbs: ["update"]
+              - apiGroups: ["certificates.k8s.io"]
+                resources: ["signers"]
+                resourceNames: ["issuers.cert-manager.io/*", "clusterissuers.cert-manager.io/*"]
+                verbs: ["sign"]
+              - apiGroups: ["authorization.k8s.io"]
+                resources: ["subjectaccessreviews"]
+                verbs: ["create"]
+
+              # Leader election roles. Although leader election only operates on
+              # one single namespace (often kube-system), we have to use a
+              # ClusterRole due to schema.yaml limitations:
+              # https://github.com/GoogleCloudPlatform/marketplace-k8s-app-tools/issues/564
+              - apiGroups: [""]
+                verbs: [create]
+                resources: [configmaps]
+              - apiGroups: [""]
+                resources: [configmaps]
+                verbs: [get, list, watch, update, patch, delete]
+                resourceNames: ["cert-manager-controller"]
+              - apiGroups: [""]
+                resources: [configmaps/status]
+                verbs: [get, update, patch]
+              - apiGroups: [coordination.k8s.io]
+                resources: [leases]
+                verbs: [create]
+              - apiGroups: [coordination.k8s.io]
+                resources: [leases]
+                verbs: [get, update, patch]
+                resourceNames: ["cert-manager-controller"]
+
   cert-manager.webhook.serviceAccount.name:
     type: string
     title: Name of the Kubernetes service account for the cert-manager webhook
@@ -274,19 +297,10 @@ properties:
           - type: Role
             rulesType: CUSTOM
             rules:
-              # Leader election roles.
-              - apiGroups: [""]
-                resources: [configmaps]
-                verbs: [get, list, watch, create, update, patch, delete]
-              - apiGroups: [""]
-                resources: [configmaps/status]
-                verbs: [get, update, patch]
               - apiGroups: [""]
                 resources: [events]
                 verbs: [create]
-              - apiGroups: [coordination.k8s.io]
-                resources: [leases]
-                verbs: [create, get, list, update]
+
           - type: ClusterRole
             rulesType: CUSTOM
             rules:
@@ -312,30 +326,36 @@ properties:
                 resources: ["configmaps"]
                 verbs: ["get", "create", "update", "patch"]
 
-  google-cas-issuer.serviceAccount.name:
-    type: string
-    title: Name of the Kubernetes service account for the google-cas-issuer
-    x-google-marketplace:
-      type: SERVICE_ACCOUNT
-      serviceAccount:
-        description: Service account used by google-cas-issuer
-        roles:
-          - type: Role
-            rulesType: CUSTOM
-            rules:
-              # Leader election roles.
+              # Leader election roles. Although leader election only operates on
+              # one single namespace (often kube-system), we have to use a
+              # ClusterRole due to schema.yaml limitations:
+              # https://github.com/GoogleCloudPlatform/marketplace-k8s-app-tools/issues/564
+              - apiGroups: [""]
+                verbs: [create]
+                resources: [configmaps]
               - apiGroups: [""]
                 resources: [configmaps]
-                verbs: [get, list, watch, create, update, patch, delete]
+                verbs: [get, list, watch, update, patch, delete]
+                resourceNames: ["cert-manager-cainjector-leader-election", "cert-manager-cainjector-leader-election-core"]
               - apiGroups: [""]
                 resources: [configmaps/status]
                 verbs: [get, update, patch]
-              - apiGroups: [""]
-                resources: [events]
+              - apiGroups: [coordination.k8s.io]
+                resources: [leases]
                 verbs: [create]
               - apiGroups: [coordination.k8s.io]
                 resources: [leases]
-                verbs: [create, get, list, update]
+                verbs: [get, update, patch]
+                resourceNames: ["cert-manager-cainjector-leader-election", "cert-manager-cainjector-leader-election-core"]
+
+  google-cas-issuer.serviceAccount.name:
+    type: string
+    title: Name of the Kubernetes service account for the google-cas-issuer
+    x-google-marketplace:
+      type: SERVICE_ACCOUNT
+      serviceAccount:
+        description: Service account used by google-cas-issuer
+        roles:
           - type: ClusterRole
             rulesType: CUSTOM
             rules:
@@ -364,6 +384,28 @@ properties:
                 resources: [certificaterequests/status]
                 verbs: [get, patch, update]
 
+              # Leader election roles. Although leader election only operates on
+              # one single namespace (often kube-system), we have to use a
+              # ClusterRole due to schema.yaml limitations:
+              # https://github.com/GoogleCloudPlatform/marketplace-k8s-app-tools/issues/564
+              - apiGroups: [""]
+                verbs: [create]
+                resources: [configmaps]
+              - apiGroups: [""]
+                resources: [configmaps]
+                verbs: [get, list, watch, update, patch, delete]
+                resourceNames: ["cm-google-cas-issuer"]
+              - apiGroups: [""]
+                resources: [configmaps/status]
+                verbs: [get, update, patch]
+              - apiGroups: [coordination.k8s.io]
+                resources: [leases]
+                verbs: [create]
+              - apiGroups: [coordination.k8s.io]
+                resources: [leases]
+                verbs: [get, update, patch]
+                resourceNames: ["cm-google-cas-issuer"]
+
   preflight.serviceAccount.name:
     type: string
     title: Name of the Kubernetes service account for the Jetstack Secure agent