Skip to content

Commit f007012

Browse files
authored
Merge pull request kubernetes#130700 from pohly/dra-kubeletplugin-helper
DRA kubeletplugin: turn helper into wrapper
2 parents 157f42b + 1057407 commit f007012

File tree

14 files changed

+867
-346
lines changed

14 files changed

+867
-346
lines changed

staging/src/k8s.io/dynamic-resource-allocation/kubeletplugin/doc.go

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,4 +16,52 @@ limitations under the License.
1616

1717
// Package kubeletplugin provides helper functions for running a dynamic
1818
// resource allocation kubelet plugin.
19+
//
20+
// A DRA driver using this package can be deployed as a DaemonSet on suitable
21+
// nodes. Node labeling, for example through NFD
22+
// (https://github.com/kubernetes-sigs/node-feature-discovery), can be used
23+
// to run the driver only on nodes which have the necessary hardware.
24+
//
25+
// The service account of the DaemonSet must have sufficient RBAC permissions
26+
// to read ResourceClaims and to create and update ResourceSlices, if
27+
// the driver intends to publish per-node ResourceSlices. It is good
28+
// security practice (but not required) to limit access to ResourceSlices
29+
// associated with the node a specific Pod is running on. This can be done
30+
// with a Validating Admission Policy (VAP). For more information,
31+
// see the deployment of the DRA example driver
32+
// (https://github.com/kubernetes-sigs/dra-example-driver/tree/main/deployments/helm/dra-example-driver/templates).
33+
//
34+
// Traditionally, the kubelet has not supported rolling updates of plugins.
35+
// Therefore the DaemonSet must not set `maxSurge` to a value larger than
36+
// zero. With the default `maxSurge: 0`, updating the DaemonSet of the driver
37+
// will first shut down the old driver Pod, then start the replacement.
38+
//
39+
// This leads to a short downtime for operations that need the driver:
40+
// - Pods cannot start unless the claims they depend on were already
41+
// prepared for use.
42+
// - Cleanup after the last pod which used a claim gets delayed
43+
// until the driver is available again. The pod is not marked
44+
// as terminated. This prevents reusing the resources used by
45+
// the pod for other pods.
46+
// - Running pods are *not* affected as far as Kubernetes is
47+
// concerned. However, a DRA driver might provide required runtime
48+
// services. Vendors need to document this.
49+
//
50+
// Note that the second point also means that draining a node should
51+
// first evict normal pods, then the driver DaemonSet Pod.
52+
//
53+
// Starting with Kubernetes 1.33, the kubelet supports rolling updates
54+
// such that old and new Pod run at the same time for a short while
55+
// and hand over work gracefully, with no downtime.
56+
// However, there is no mechanism for determining in advance whether
57+
// the node the DaemonSet runs on supports that. Trying
58+
// to do a rolling update with a kubelet which does not support it yet
59+
// will fail because shutting down the old Pod unregisters the driver
60+
// even though the new Pod is running. See https://github.com/kubernetes/kubernetes/pull/129832
61+
// for details (TODO: link to doc after merging instead).
62+
//
63+
// A DRA driver can either require 1.33 as minimal Kubernetes version or
64+
// provide two variants of its DaemonSet deployment. In the variant with
65+
// support for rolling updates, `maxSurge` can be set to a non-zero
66+
// value. Administrators have to be careful about running the right variant.
1967
package kubeletplugin

0 commit comments

Comments
 (0)