addresses kubeadm upgrade review comments

mhausenblas · mhausenblas · commit f4ce844fcbec · 2017-09-16T11:27:12.000+02:00
CC: @luxas
diff --git a/docs/tasks/administer-cluster/kubeadm-upgrade-1-8.md b/docs/tasks/administer-cluster/kubeadm-upgrade-1-8.md
@@ -4,12 +4,12 @@ approvers:
 - luxas
 - roberthbailey
 - jbeda
-title: Upgrading kubeadm clusters
+title: Upgrading kubeadm clusters from 1.7 to 1.8
 ---
 
 {% capture overview %}
 
-This guide is for upgrading `kubeadm` clusters from version 1.7.x to 1.8.x.
+This guide is for upgrading `kubeadm` clusters from version 1.7.x to 1.8.x, as well as 1.7.x to 1.7.y and 1.8.x to 1.8.y where `y > x`.
 See also [upgrading kubeadm clusters from 1.6 to 1.7](/docs/tasks/administer-cluster/kubeadm-upgrade-1-7/) if you're on a 1.6 cluster currently.
 
 {% endcapture %}
@@ -41,95 +41,140 @@ $ export ARCH=amd64 # or: arm, arm64, ppc64le, s390x
 $ curl -sSL https://dl.k8s.io/release/${VERSION}/bin/linux/${ARCH}/kubeadm > /usr/bin/kubeadm
 ```
 
-1. On the master node, run the following:
+2. If this the first time you use `kubeadm upgrade`, in order to preserve the configuration for future upgrades, do:
+
+Note that for below you will need to recall what CLI args you passed to `kubeadm init` the first time.
+
+If you used flags, do:
+
+```shell
+$ kubeadm config upload from-flags [flags]
+```
+
+Where `flags` can be empty.
+
+If you used a config file, do:
+
+```shell
+$ kubeadm config upload from-file --config [config]
+```
+
+Where the `config` is mandatory.
+
+3. On the master node, run the following:
 
 ```shell
 $ kubeadm upgrade plan
+[preflight] Running pre-flight checks
 [upgrade] Making sure the cluster is healthy:
 [upgrade/health] Checking API Server health: Healthy
 [upgrade/health] Checking Node health: All Nodes are healthy
-[upgrade/health] Checking if control plane is Static Pod-hosted or Self-Hosted: Static Pod-hosted.
-[upgrade/health] NOTE: kubeadm will upgrade your Static Pod-hosted control plane to a Self-Hosted one when upgrading if --feature-gates=SelfHosting=true is set (which is the default)
-[upgrade/health] If you strictly want to continue using a Static Pod-hosted control plane, set --feature-gates=SelfHosting=true when running 'kubeadm upgrade apply'
-[upgrade/health] Checking Static Pod manifests exists on disk: All required Static Pod manifests exist on disk
-[upgrade] Making sure the configuration is correct:
-[upgrade/config] Reading configuration from the cluster (you can get this with 'kubectl -n kube-system get cm kubeadm-config -oyaml')
+[upgrade/health] Checking Static Pod manifests exists on disk: All manifests exist on disk
+[upgrade/config] Making sure the configuration is correct:
+[upgrade/config] Reading configuration from the cluster...
+[upgrade/config] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
 [upgrade] Fetching available versions to upgrade to:
 [upgrade/versions] Cluster version: v1.7.1
 [upgrade/versions] kubeadm version: v1.8.0
-[upgrade/versions] Latest stable version: v1.7.3
-[upgrade/versions] Latest version in the v1.7 series: v1.7.3
+[upgrade/versions] Latest stable version: v1.8.0
+[upgrade/versions] Latest version in the v1.7 series: v1.7.6
 
-Components that must be upgraded manually after you've upgraded the control plane with `kubeadm upgrade apply`:
+Components that must be upgraded manually after you've upgraded the control plane with 'kubeadm upgrade apply':
 COMPONENT   CURRENT      AVAILABLE
-Kubelet     1 x v1.7.0   v1.7.3
+Kubelet     1 x v1.7.1   v1.7.6
 
 Upgrade to the latest version in the v1.7 series:
 
 COMPONENT            CURRENT   AVAILABLE
-API Server           v1.7.1    v1.7.3
-Controller Manager   v1.7.1    v1.7.3
-Scheduler            v1.7.1    v1.7.3
-Kube Proxy           v1.7.1    v1.7.3
+API Server           v1.7.1    v1.7.6
+Controller Manager   v1.7.1    v1.7.6
+Scheduler            v1.7.1    v1.7.6
+Kube Proxy           v1.7.1    v1.7.6
+Kube DNS             1.14.4    1.14.4
+
+You can now apply the upgrade by executing the following command:
+
+	kubeadm upgrade apply v1.7.6
+
+_____________________________________________________________________
+
+Components that must be upgraded manually after you've upgraded the control plane with 'kubeadm upgrade apply':
+COMPONENT   CURRENT      AVAILABLE
+Kubelet     1 x v1.7.1   v1.8.0
+
+Upgrade to the latest experimental version:
+
+COMPONENT            CURRENT   AVAILABLE
+API Server           v1.7.1    v1.8.0
+Controller Manager   v1.7.1    v1.8.0
+Scheduler            v1.7.1    v1.8.0
+Kube Proxy           v1.7.1    v1.8.0
 Kube DNS             1.14.4    1.14.4
 
 You can now apply the upgrade by executing the following command:
 
-	kubeadm upgrade apply --version v1.7.3
+	kubeadm upgrade apply v1.8.0
+
+Note: Before you do can perform this upgrade, you have to update kubeadm to v1.8.0
+
+_____________________________________________________________________
 ```
 
 The `kubeadm upgrade plan` checks that your cluster is in an upgradeable state and fetches the versions available to upgrade to in an user-friendly way.
 
-1. Pick a version to upgrade to and run, for example, `kubeadm upgrade apply` as follows:
+4. Pick a version to upgrade to and run, for example, `kubeadm upgrade apply` as follows:
 
 ```shell
-$ kubeadm upgrade apply --version v1.8.0
+$ kubeadm upgrade apply v1.8.0
+[preflight] Running pre-flight checks
 [upgrade] Making sure the cluster is healthy:
 [upgrade/health] Checking API Server health: Healthy
 [upgrade/health] Checking Node health: All Nodes are healthy
-[upgrade/health] Checking if control plane is Static Pod-hosted or Self-Hosted: Static Pod-hosted.
-[upgrade/health] NOTE: kubeadm will upgrade your Static Pod-hosted control plane to a Self-Hosted one when upgrading if --feature-gates=SelfHosting=true is set (which is the default)
-[upgrade/health] If you strictly want to continue using a Static Pod-hosted control plane, set --feature-gates=SelfHosting=true when running 'kubeadm upgrade apply'
-[upgrade/health] Checking Static Pod manifests exists on disk: All required Static Pod manifests exist on disk
-[upgrade] Making sure the configuration is correct:
-[upgrade/config] Reading configuration from the cluster (you can get this with 'kubectl -n kube-system get cm kubeadm-config -oyaml')
+[upgrade/health] Checking Static Pod manifests exists on disk: All manifests exist on disk
+[upgrade/config] Making sure the configuration is correct:
+[upgrade/config] Reading configuration from the cluster...
+[upgrade/config] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
 [upgrade/version] You have chosen to upgrade to version "v1.8.0"
 [upgrade/versions] Cluster version: v1.7.1
 [upgrade/versions] kubeadm version: v1.8.0
-[upgrade/confirm] Are you sure you want to proceed with the upgrade? [y/N]: Y
 [upgrade/prepull] Will prepull images for components [kube-apiserver kube-controller-manager kube-scheduler]
 [upgrade/prepull] Prepulling image for component kube-scheduler.
 [upgrade/prepull] Prepulling image for component kube-apiserver.
 [upgrade/prepull] Prepulling image for component kube-controller-manager.
-[upgrade/prepull] Prepulled image for component kube-scheduler.
+[apiclient] Found 0 Pods for label selector k8s-app=upgrade-prepull-kube-scheduler
+[apiclient] Found 1 Pods for label selector k8s-app=upgrade-prepull-kube-scheduler
+[apiclient] Found 1 Pods for label selector k8s-app=upgrade-prepull-kube-apiserver
+[apiclient] Found 1 Pods for label selector k8s-app=upgrade-prepull-kube-controller-manager
 [upgrade/prepull] Prepulled image for component kube-apiserver.
 [upgrade/prepull] Prepulled image for component kube-controller-manager.
+[upgrade/prepull] Prepulled image for component kube-scheduler.
 [upgrade/prepull] Successfully prepulled the images for all the control plane components
 [upgrade/apply] Upgrading your Static Pod-hosted control plane to version "v1.8.0"...
-[upgrade/staticpods] Wrote upgraded Static Pod manifests to "/tmp/kubeadm-upgrade830923296"
-[upgrade/staticpods] Moved upgraded manifest to "/etc/kubernetes/manifests/kube-apiserver.yaml" and backuped old manifest to "/tmp/kubeadm-upgrade830923296/old-manifests/kube-apiserver.yaml"
+[upgrade/staticpods] Writing upgraded Static Pod manifests to "/etc/kubernetes/tmp/kubeadm-upgraded-manifests432902769"
+[controlplane] Wrote Static Pod manifest for component kube-apiserver to "/etc/kubernetes/tmp/kubeadm-upgraded-manifests432902769/kube-apiserver.yaml"
+[controlplane] Wrote Static Pod manifest for component kube-controller-manager to "/etc/kubernetes/tmp/kubeadm-upgraded-manifests432902769/kube-controller-manager.yaml"
+[controlplane] Wrote Static Pod manifest for component kube-scheduler to "/etc/kubernetes/tmp/kubeadm-upgraded-manifests432902769/kube-scheduler.yaml"
+[upgrade/staticpods] Moved upgraded manifest to "/etc/kubernetes/manifests/kube-apiserver.yaml" and backed up old manifest to "/etc/kubernetes/tmp/kubeadm-backup-manifests155856668/kube-apiserver.yaml"
 [upgrade/staticpods] Waiting for the kubelet to restart the component
 [apiclient] Found 1 Pods for label selector component=kube-apiserver
 [upgrade/staticpods] Component "kube-apiserver" upgraded successfully!
-[upgrade/staticpods] Moved upgraded manifest to "/etc/kubernetes/manifests/kube-controller-manager.yaml" and backuped old manifest to "/tmp/kubeadm-upgrade830923296/old-manifests/kube-controller-manager.yaml"
+[upgrade/staticpods] Moved upgraded manifest to "/etc/kubernetes/manifests/kube-controller-manager.yaml" and backed up old manifest to "/etc/kubernetes/tmp/kubeadm-backup-manifests155856668/kube-controller-manager.yaml"
 [upgrade/staticpods] Waiting for the kubelet to restart the component
 [apiclient] Found 1 Pods for label selector component=kube-controller-manager
 [upgrade/staticpods] Component "kube-controller-manager" upgraded successfully!
-[upgrade/staticpods] Moved upgraded manifest to "/etc/kubernetes/manifests/kube-scheduler.yaml" and backuped old manifest to "/tmp/kubeadm-upgrade830923296/old-manifests/kube-scheduler.yaml"
+[upgrade/staticpods] Moved upgraded manifest to "/etc/kubernetes/manifests/kube-scheduler.yaml" and backed up old manifest to "/etc/kubernetes/tmp/kubeadm-backup-manifests155856668/kube-scheduler.yaml"
 [upgrade/staticpods] Waiting for the kubelet to restart the component
 [apiclient] Found 1 Pods for label selector component=kube-scheduler
-[apiclient] Found 0 Pods for label selector component=kube-scheduler
-[apiclient] Found 1 Pods for label selector component=kube-scheduler
 [upgrade/staticpods] Component "kube-scheduler" upgraded successfully!
-[apiclient] Found 0 Pods for label selector k8s-app=self-hosted-kube-apiserver
-[apiclient] Found 1 Pods for label selector k8s-app=self-hosted-kube-apiserver
-[apiclient] Found 0 Pods for label selector k8s-app=self-hosted-kube-controller-manager
-[apiclient] Found 1 Pods for label selector k8s-app=self-hosted-kube-controller-manager
-[apiclient] Found 0 Pods for label selector k8s-app=self-hosted-kube-scheduler
-[apiclient] Found 1 Pods for label selector k8s-app=self-hosted-kube-scheduler
-[apiconfig] Created RBAC rules
-[addons] Applied essential addon: kube-proxy
+[uploadconfig] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
+[bootstraptoken] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
+[bootstraptoken] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
 [addons] Applied essential addon: kube-dns
+[addons] Applied essential addon: kube-proxy
+
+[upgrade/successful] SUCCESS! Your cluster was upgraded to "v1.8.0". Enjoy!
+
+[upgrade/kubelet] Now that your control plane is upgraded, please proceed with upgrading your kubelets in turn.
 ```
 
 `kubeadm upgrade apply` does the following:
@@ -143,7 +188,7 @@ $ kubeadm upgrade apply --version v1.8.0
 - It upgrades the control plane components or rollbacks if any of them fails to come up.
 - It applies the new `kube-dns` and `kube-proxy` manifests and enforces that all necessary RBAC rules are created.
 
-1. Manually upgrade your Software Defined Network (SDN).
+5. Manually upgrade your Software Defined Network (SDN).
 
    Your Container Network Interface (CNI) provider might have its own upgrade instructions to follow now.
    Check the [addons](/docs/concepts/cluster-administration/addons/) page to
@@ -161,29 +206,35 @@ $ kubectl cordon $WORKER
 $ kubectl drain $WORKER
 ```
 
-1. Upgrade the `kubelet` version on the `$WORKER` node, either by using a Linux distribution-specific package manager such as `apt-get` or `yum` or manually as described in the following:
+2. Upgrade the `kubelet` version on the `$WORKER` node by using a Linux distribution-specific package manager:
+
+If the node is running a Debian-based distro such as Ubuntu, run:
+
+```shell
+$ apt-get update
+$ apt-get install -y kubelet
+```
+
+If the node is running CentOS or the like, run:
 
 ```shell
-$ sudo systemctl stop kubelet
-$ curl -s -L -o kubelet \
-  https://storage.googleapis.com/kubernetes-release/release/v1.8.0/bin/linux/amd64/kubelet
-$ chmod +x kubectl && sudo mv kubelet /usr/local/bin/
-$ sudo systemctl start kubelet
+$ yum update
+$ yum install -y kubelet
 ```
 
-Now, the new version of the `kubelet` should be running on the `$WORKER` node. Verify this using the following command:
+Now the new version of the `kubelet` should be running on the `$WORKER` node. Verify this using the following command:
 
 ```shell
 $ systemctl status kubelet
 ```
 
-1. Bring the `$WORKER` node back online by marking it schedulable:
+3. Bring the `$WORKER` node back online by marking it schedulable:
 
 ```shell
 $ kubectl uncordon $WORKER
 ```
 
-1. After upgrading `kubelet` on each worker node in your cluster, verify that all nodes are available again by executing the following (from anywhere, for example, from outside the cluster):
+4. After upgrading `kubelet` on each worker node in your cluster, verify that all nodes are available again by executing the following (from anywhere, for example, from outside the cluster):
 
 ```shell
 $ kubectl get nodes
