|
| 1 | +--- |
| 2 | +layout: blog |
| 3 | +title: 'Kubernetes 1.30: Multi-Webhook and Modular Authorization Made Much Easier' |
| 4 | +date: 2024-04-26 |
| 5 | +slug: multi-webhook-and-modular-authorization-made-much-easier |
| 6 | +--- |
| 7 | + |
| 8 | +**Authors:** [Rita Zhang](https://github.com/ritazh) (Microsoft), [Jordan |
| 9 | +Liggitt](https://github.com/liggitt) (Google), [Nabarun |
| 10 | +Pal](https://github.com/palnabarun) (VMware), [Leigh |
| 11 | +Capili](https://github.com/stealthybox) (VMware) |
| 12 | + |
| 13 | +With Kubernetes 1.30, we (SIG Auth) are moving Structured Authorization |
| 14 | +Configuration to beta. |
| 15 | + |
| 16 | +Today's article is about _authorization_: deciding what someone can and cannot |
| 17 | +access. Check a previous article from yesterday to find about what's new in |
| 18 | +Kubernetes v1.30 around _authentication_ (finding out who's performing a task, |
| 19 | +and checking that they are who they say they are). |
| 20 | + |
| 21 | +## Introduction |
| 22 | +Kubernetes continues to evolve to meet the intricate requirements of system |
| 23 | +administrators and developers alike. A critical aspect of Kubernetes that |
| 24 | +ensures the security and integrity of the cluster is the API server |
| 25 | +authorization. Until recently, the configuration of the authorization chain in |
| 26 | +kube-apiserver was somewhat rigid, limited to a set of command-line flags and |
| 27 | +allowing only a single webhook in the authorization chain. This approach, while |
| 28 | +functional, restricted the flexibility needed by cluster administrators to |
| 29 | +define complex, fine-grained authorization policies. The latest Structured |
| 30 | +Authorization Configuration feature ([KEP-3221](https://kep.k8s.io/3221)) aims |
| 31 | +to revolutionize this aspect by introducing a more structured and versatile way |
| 32 | +to configure the authorization chain, focusing on enabling multiple webhooks and |
| 33 | +providing explicit control mechanisms. |
| 34 | + |
| 35 | +## The Need for Improvement |
| 36 | +Cluster administrators have long sought the ability to specify multiple |
| 37 | +authorization webhooks within the API Server handler chain and have control over |
| 38 | +detailed behavior like timeout and failure policy for each webhook. This need |
| 39 | +arises from the desire to create layered security policies, where requests can |
| 40 | +be validated against multiple criteria or sets of rules in a specific order. The |
| 41 | +previous limitations also made it difficult to dynamically configure the |
| 42 | +authorizer chain, leaving no room to manage complex authorization scenarios |
| 43 | +efficiently. |
| 44 | + |
| 45 | +The [Structured Authorization Configuration |
| 46 | +feature](/docs/reference/access-authn-authz/authorization/#configuring-the-api-server-using-an-authorization-config-file) |
| 47 | +addresses these limitations by introducing a configuration file format to |
| 48 | +configure the Kubernetes API Server Authorization chain. This format allows |
| 49 | +specifying multiple webhooks in the authorization chain (all other authorization |
| 50 | +types are specified no more than once). Each webhook authorizer has well-defined |
| 51 | +parameters, including timeout settings, failure policies, and conditions for |
| 52 | +invocation with [CEL](/docs/reference/using-api/cel/) rules to pre-filter |
| 53 | +requests before they are dispatched to webhooks, helping you prevent unnecessary |
| 54 | +invocations. The configuration also supports automatic reloading, ensuring |
| 55 | +changes can be applied dynamically without restarting the kube-apiserver. This |
| 56 | +feature addresses current limitations and opens up new possibilities for |
| 57 | +securing and managing Kubernetes clusters more effectively. |
| 58 | + |
| 59 | +## Sample Configurations |
| 60 | +Here is a sample structured authorization configuration along with descriptions |
| 61 | +for all fields, their defaults, and possible values. |
| 62 | + |
| 63 | +```yaml |
| 64 | +apiVersion: apiserver.config.k8s.io/v1beta1 |
| 65 | +kind: AuthorizationConfiguration |
| 66 | +authorizers: |
| 67 | + - type: Webhook |
| 68 | + # Name used to describe the authorizer |
| 69 | + # This is explicitly used in monitoring machinery for metrics |
| 70 | + # Note: |
| 71 | + # - Validation for this field is similar to how K8s labels are validated today. |
| 72 | + # Required, with no default |
| 73 | + name: webhook |
| 74 | + webhook: |
| 75 | + # The duration to cache 'authorized' responses from the webhook |
| 76 | + # authorizer. |
| 77 | + # Same as setting `--authorization-webhook-cache-authorized-ttl` flag |
| 78 | + # Default: 5m0s |
| 79 | + authorizedTTL: 30s |
| 80 | + # The duration to cache 'unauthorized' responses from the webhook |
| 81 | + # authorizer. |
| 82 | + # Same as setting `--authorization-webhook-cache-unauthorized-ttl` flag |
| 83 | + # Default: 30s |
| 84 | + unauthorizedTTL: 30s |
| 85 | + # Timeout for the webhook request |
| 86 | + # Maximum allowed is 30s. |
| 87 | + # Required, with no default. |
| 88 | + timeout: 3s |
| 89 | + # The API version of the authorization.k8s.io SubjectAccessReview to |
| 90 | + # send to and expect from the webhook. |
| 91 | + # Same as setting `--authorization-webhook-version` flag |
| 92 | + # Required, with no default |
| 93 | + # Valid values: v1beta1, v1 |
| 94 | + subjectAccessReviewVersion: v1 |
| 95 | + # MatchConditionSubjectAccessReviewVersion specifies the SubjectAccessReview |
| 96 | + # version the CEL expressions are evaluated against |
| 97 | + # Valid values: v1 |
| 98 | + # Required, no default value |
| 99 | + matchConditionSubjectAccessReviewVersion: v1 |
| 100 | + # Controls the authorization decision when a webhook request fails to |
| 101 | + # complete or returns a malformed response or errors evaluating |
| 102 | + # matchConditions. |
| 103 | + # Valid values: |
| 104 | + # - NoOpinion: continue to subsequent authorizers to see if one of |
| 105 | + # them allows the request |
| 106 | + # - Deny: reject the request without consulting subsequent authorizers |
| 107 | + # Required, with no default. |
| 108 | + failurePolicy: Deny |
| 109 | + connectionInfo: |
| 110 | + # Controls how the webhook should communicate with the server. |
| 111 | + # Valid values: |
| 112 | + # - KubeConfig: use the file specified in kubeConfigFile to locate the |
| 113 | + # server. |
| 114 | + # - InClusterConfig: use the in-cluster configuration to call the |
| 115 | + # SubjectAccessReview API hosted by kube-apiserver. This mode is not |
| 116 | + # allowed for kube-apiserver. |
| 117 | + type: KubeConfig |
| 118 | + # Path to KubeConfigFile for connection info |
| 119 | + # Required, if connectionInfo.Type is KubeConfig |
| 120 | + kubeConfigFile: /kube-system-authz-webhook.yaml |
| 121 | + # matchConditions is a list of conditions that must be met for a request to be sent to this |
| 122 | + # webhook. An empty list of matchConditions matches all requests. |
| 123 | + # There are a maximum of 64 match conditions allowed. |
| 124 | + # |
| 125 | + # The exact matching logic is (in order): |
| 126 | + # 1. If at least one matchCondition evaluates to FALSE, then the webhook is skipped. |
| 127 | + # 2. If ALL matchConditions evaluate to TRUE, then the webhook is called. |
| 128 | + # 3. If at least one matchCondition evaluates to an error (but none are FALSE): |
| 129 | + # - If failurePolicy=Deny, then the webhook rejects the request |
| 130 | + # - If failurePolicy=NoOpinion, then the error is ignored and the webhook is skipped |
| 131 | + matchConditions: |
| 132 | + # expression represents the expression which will be evaluated by CEL. Must evaluate to bool. |
| 133 | + # CEL expressions have access to the contents of the SubjectAccessReview in v1 version. |
| 134 | + # If version specified by subjectAccessReviewVersion in the request variable is v1beta1, |
| 135 | + # the contents would be converted to the v1 version before evaluating the CEL expression. |
| 136 | + # |
| 137 | + # Documentation on CEL: https://kubernetes.io/docs/reference/using-api/cel/ |
| 138 | + # |
| 139 | + # only send resource requests to the webhook |
| 140 | + - expression: has(request.resourceAttributes) |
| 141 | + # only intercept requests to kube-system |
| 142 | + - expression: request.resourceAttributes.namespace == 'kube-system' |
| 143 | + # don't intercept requests from kube-system service accounts |
| 144 | + - expression: !('system:serviceaccounts:kube-system' in request.user.groups) |
| 145 | + - type: Node |
| 146 | + name: node |
| 147 | + - type: RBAC |
| 148 | + name: rbac |
| 149 | + - type: Webhook |
| 150 | + name: in-cluster-authorizer |
| 151 | + webhook: |
| 152 | + authorizedTTL: 5m |
| 153 | + unauthorizedTTL: 30s |
| 154 | + timeout: 3s |
| 155 | + subjectAccessReviewVersion: v1 |
| 156 | + failurePolicy: NoOpinion |
| 157 | + connectionInfo: |
| 158 | + type: InClusterConfig |
| 159 | +``` |
| 160 | +
|
| 161 | +The following configuration examples illustrate real-world scenarios that need |
| 162 | +the ability to specify multiple webhooks with distinct settings, precedence |
| 163 | +order, and failure modes. |
| 164 | +
|
| 165 | +### Protecting Installed CRDs |
| 166 | +Ensuring of Custom Resource Definitions (CRDs) availability at cluster startup |
| 167 | +has been a key demand. One of the blockers of having a controller reconcile |
| 168 | +those CRDs is having a protection mechanism for them, which can be achieved |
| 169 | +through multiple authorization webhooks. This was not possible before as |
| 170 | +specifying multiple authorization webhooks in the Kubernetes API Server |
| 171 | +authorization chain was simply not possible. Now, with the Structured |
| 172 | +Authorization Configuration feature, administrators can specify multiple |
| 173 | +webhooks, offering a solution where RBAC falls short, especially when denying |
| 174 | +permissions to 'non-system' users for certain CRDs. |
| 175 | +
|
| 176 | +Assuming the following for this scenario: |
| 177 | +- The "protected" CRDs are installed. |
| 178 | +- They can only be modified by users in the group `admin`. |
| 179 | + |
| 180 | +```yaml |
| 181 | +apiVersion: apiserver.config.k8s.io/v1beta1 |
| 182 | +kind: AuthorizationConfiguration |
| 183 | +authorizers: |
| 184 | + - type: Webhook |
| 185 | + name: system-crd-protector |
| 186 | + webhook: |
| 187 | + unauthorizedTTL: 30s |
| 188 | + timeout: 3s |
| 189 | + subjectAccessReviewVersion: v1 |
| 190 | + matchConditionSubjectAccessReviewVersion: v1 |
| 191 | + failurePolicy: Deny |
| 192 | + connectionInfo: |
| 193 | + type: KubeConfig |
| 194 | + kubeConfigFile: /files/kube-system-authz-webhook.yaml |
| 195 | + matchConditions: |
| 196 | + # only send resource requests to the webhook |
| 197 | + - expression: has(request.resourceAttributes) |
| 198 | + # only intercept requests for CRDs |
| 199 | + - expression: request.resourceAttributes.resource.resource = "customresourcedefinitions" |
| 200 | + - expression: request.resourceAttributes.resource.group = "" |
| 201 | + # only intercept update, patch, delete, or deletecollection requests |
| 202 | + - expression: request.resourceAttributes.verb in ['update', 'patch', 'delete','deletecollection'] |
| 203 | + - type: Node |
| 204 | + - type: RBAC |
| 205 | +``` |
| 206 | + |
| 207 | +### Preventing unnecessarily nested webhooks |
| 208 | +A system administrator wants to apply specific validations to requests before |
| 209 | +handing them off to webhooks using frameworks like Open Policy Agent. In the |
| 210 | +past, this would require running nested webhooks within the one added to the |
| 211 | +authorization chain to achieve the desired result. The Structured Authorization |
| 212 | +Configuration feature simplifies this process, offering a structured API to |
| 213 | +selectively trigger additional webhooks when needed. It also enables |
| 214 | +administrators to set distinct failure policies for each webhook, ensuring more |
| 215 | +consistent and predictable responses. |
| 216 | + |
| 217 | +```yaml |
| 218 | +apiVersion: apiserver.config.k8s.io/v1beta1 |
| 219 | +kind: AuthorizationConfiguration |
| 220 | +authorizers: |
| 221 | + - type: Webhook |
| 222 | + name: system-crd-protector |
| 223 | + webhook: |
| 224 | + unauthorizedTTL: 30s |
| 225 | + timeout: 3s |
| 226 | + subjectAccessReviewVersion: v1 |
| 227 | + matchConditionSubjectAccessReviewVersion: v1 |
| 228 | + failurePolicy: Deny |
| 229 | + connectionInfo: |
| 230 | + type: KubeConfig |
| 231 | + kubeConfigFile: /files/kube-system-authz-webhook.yaml |
| 232 | + matchConditions: |
| 233 | + # only send resource requests to the webhook |
| 234 | + - expression: has(request.resourceAttributes) |
| 235 | + # only intercept requests for CRDs |
| 236 | + - expression: request.resourceAttributes.resource.resource = "customresourcedefinitions" |
| 237 | + - expression: request.resourceAttributes.resource.group = "" |
| 238 | + # only intercept update, patch, delete, or deletecollection requests |
| 239 | + - expression: request.resourceAttributes.verb in ['update', 'patch', 'delete','deletecollection'] |
| 240 | + - type: Node |
| 241 | + - type: RBAC |
| 242 | + - name: opa |
| 243 | + type: Webhook |
| 244 | + webhook: |
| 245 | + unauthorizedTTL: 30s |
| 246 | + timeout: 3s |
| 247 | + subjectAccessReviewVersion: v1 |
| 248 | + matchConditionSubjectAccessReviewVersion: v1 |
| 249 | + failurePolicy: Deny |
| 250 | + connectionInfo: |
| 251 | + type: KubeConfig |
| 252 | + kubeConfigFile: /files/opa-default-authz-webhook.yaml |
| 253 | + matchConditions: |
| 254 | + # only send resource requests to the webhook |
| 255 | + - expression: has(request.resourceAttributes) |
| 256 | + # only intercept requests to default namespace |
| 257 | + - expression: request.resourceAttributes.namespace == 'default' |
| 258 | + # don't intercept requests from default service accounts |
| 259 | + - expression: !('system:serviceaccounts:default' in request.user.groups) |
| 260 | +``` |
| 261 | + |
| 262 | +## What's next? |
| 263 | +From Kubernetes 1.30, the feature is in beta and enabled by default. For |
| 264 | +Kubernetes v1.31, we expect the feature to stay in beta while we get more |
| 265 | +feedback from users. Once it is ready for GA, the feature flag will be removed, |
| 266 | +and the configuration file version will be promoted to v1. |
| 267 | + |
| 268 | +Learn more about this feature on the [structured authorization |
| 269 | +configuration](/docs/reference/access-authn-authz/authorization/#configuring-the-api-server-using-an-authorization-config-file) |
| 270 | +Kubernetes doc website. You can also follow along with |
| 271 | +[KEP-3221](https://kep.k8s.io/3221) to track progress in coming Kubernetes |
| 272 | +releases. |
| 273 | + |
| 274 | +## Call to action |
| 275 | +In this post, we have covered the benefits of the Structured Authorization |
| 276 | +Configuration feature in Kubernetes v1.30 and a few sample configurations for |
| 277 | +real-world scenarios. To use this feature, you must specify the path to the |
| 278 | +authorization configuration using the `--authorization-config` command line |
| 279 | +argument. From Kubernetes 1.30, the feature is in beta and enabled by default. |
| 280 | +If you want to keep using command line flags instead of a configuration file, |
| 281 | +those will continue to work as-is. Specifying both `--authorization-config` and |
| 282 | +`--authorization-modes`/`--authorization-webhook-*` won't work. You need to drop |
| 283 | +the older flags from your kube-apiserver command. |
| 284 | + |
| 285 | +The following kind Cluster configuration sets that command argument on the |
| 286 | +APIserver to load an AuthorizationConfiguration from a file |
| 287 | +(`authorization_config.yaml`) in the files folder. Any needed kubeconfig and |
| 288 | +certificate files can also be put in the files directory. |
| 289 | +```yaml |
| 290 | +kind: Cluster |
| 291 | +apiVersion: kind.x-k8s.io/v1alpha4 |
| 292 | +featureGates: |
| 293 | + StructuredAuthorizationConfiguration: true # enabled by default in v1.30 |
| 294 | +kubeadmConfigPatches: |
| 295 | + - | |
| 296 | + kind: ClusterConfiguration |
| 297 | + metadata: |
| 298 | + name: config |
| 299 | + apiServer: |
| 300 | + extraArgs: |
| 301 | + authorization-config: "/files/authorization_config.yaml" |
| 302 | + extraVolumes: |
| 303 | + - name: files |
| 304 | + hostPath: "/files" |
| 305 | + mountPath: "/files" |
| 306 | + readOnly: true |
| 307 | +nodes: |
| 308 | +- role: control-plane |
| 309 | + extraMounts: |
| 310 | + - hostPath: files |
| 311 | + containerPath: /files |
| 312 | +``` |
| 313 | + |
| 314 | +We would love to hear your feedback on this feature. In particular, we would |
| 315 | +like feedback from Kubernetes cluster administrators and authorization webhook |
| 316 | +implementors as they build their integrations with this new API. Please reach |
| 317 | +out to us on the |
| 318 | +[#sig-auth-authorizers-dev](https://kubernetes.slack.com/archives/C05EZFX1Z2L) |
| 319 | +channel on Kubernetes Slack. |
| 320 | + |
| 321 | +## How to get involved |
| 322 | +If you are interested in helping develop this feature, sharing feedback, or |
| 323 | +participating in any other ongoing SIG Auth projects, please reach out on the |
| 324 | +[#sig-auth](https://kubernetes.slack.com/archives/C0EN96KUY) channel on |
| 325 | +Kubernetes Slack. |
| 326 | + |
| 327 | +You are also welcome to join the bi-weekly [SIG Auth |
| 328 | +meetings](https://github.com/kubernetes/community/blob/master/sig-auth/README.md#meetings) |
| 329 | +held every other Wednesday. |
| 330 | + |
| 331 | +## Acknowledgments |
| 332 | +This feature was driven by contributors from several different companies. We |
| 333 | +would like to extend a huge thank you to everyone who contributed their time and |
| 334 | +effort to make this possible. |
0 commit comments