diff --git a/keps/sig-api-machinery/5667-crd-map-key-validation/README.md b/keps/sig-api-machinery/5667-crd-map-key-validation/README.md new file mode 100644 index 00000000000..f1af1a102e7 --- /dev/null +++ b/keps/sig-api-machinery/5667-crd-map-key-validation/README.md @@ -0,0 +1,647 @@ +# KEP-5667: CRD Map Key Validation + + +- [Release Signoff Checklist](#release-signoff-checklist) +- [Summary](#summary) +- [Motivation](#motivation) + - [Goals](#goals) + - [Non-Goals](#non-goals) +- [Proposal](#proposal) + - [User Stories (Optional)](#user-stories-optional) + - [Story 1](#story-1) + - [Story 2](#story-2) + - [Notes/Constraints/Caveats (Optional)](#notesconstraintscaveats-optional) + - [Risks and Mitigations](#risks-and-mitigations) +- [Design Details](#design-details) + - [Test Plan](#test-plan) + - [Prerequisite testing updates](#prerequisite-testing-updates) + - [Unit tests](#unit-tests) + - [Integration tests](#integration-tests) + - [e2e tests](#e2e-tests) + - [Graduation Criteria](#graduation-criteria) + - [Upgrade / Downgrade Strategy](#upgrade--downgrade-strategy) + - [Version Skew Strategy](#version-skew-strategy) +- [Production Readiness Review Questionnaire](#production-readiness-review-questionnaire) + - [Feature Enablement and Rollback](#feature-enablement-and-rollback) + - [Rollout, Upgrade and Rollback Planning](#rollout-upgrade-and-rollback-planning) + - [Monitoring Requirements](#monitoring-requirements) + - [Dependencies](#dependencies) + - [Scalability](#scalability) + - [Troubleshooting](#troubleshooting) +- [Implementation History](#implementation-history) +- [Drawbacks](#drawbacks) +- [Alternatives](#alternatives) +- [Infrastructure Needed (Optional)](#infrastructure-needed-optional) + + +## Release Signoff Checklist + + + +Items marked with (R) are required *prior to targeting to a milestone / release*. + +- [ ] (R) Enhancement issue in release milestone, which links to KEP dir in [kubernetes/enhancements] (not the initial KEP PR) +- [ ] (R) KEP approvers have approved the KEP status as `implementable` +- [ ] (R) Design details are appropriately documented +- [ ] (R) Test plan is in place, giving consideration to SIG Architecture and SIG Testing input (including test refactors) + - [ ] e2e Tests for all Beta API Operations (endpoints) + - [ ] (R) Ensure GA e2e tests meet requirements for [Conformance Tests](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/conformance-tests.md) + - [ ] (R) Minimum Two Week Window for GA e2e tests to prove flake free +- [ ] (R) Graduation criteria is in place + - [ ] (R) [all GA Endpoints](https://github.com/kubernetes/community/pull/1806) must be hit by [Conformance Tests](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/conformance-tests.md) within one minor version of promotion to GA +- [ ] (R) Production readiness review completed +- [ ] (R) Production readiness review approved +- [ ] "Implementation History" section is up-to-date for milestone +- [ ] User-facing documentation has been created in [kubernetes/website], for publication to [kubernetes.io] +- [ ] Supporting documentation—e.g., additional design documents, links to mailing list discussions/SIG meetings, relevant PRs/issues, release notes + + + +[kubernetes.io]: https://kubernetes.io/ +[kubernetes/enhancements]: https://git.k8s.io/enhancements +[kubernetes/kubernetes]: https://git.k8s.io/kubernetes +[kubernetes/website]: https://git.k8s.io/website + +## Summary + +We propose adding a `x-kubernetes-property-names` extension to OpenAPI 3.0. + +This extension can be used to validate map keys. For example, to define map +of string to string where the keys and values have a max length of 32: + +```yaml +type: object +properties: + exampleMap: + type: object + x-kubernetes-property-names: + type: string + maxLength: 32 + additionalProperties: + type: string + maxLength: 32 +``` + +## Motivation + +OpenAPI 3.0 does not provide a direct way to validate map keys. + +OpenAPI 3.1 provides validation using `propertyNames`. But OpenAPI 3.1 is backward +incompatible with OpenAPI 3.0. See https://github.com/kubernetes/kubernetes/issues/134684 +for details on the backward incompatible changes made. + +Since upgrading to OpenAPI 3.1 would be a major effort and would be disruptive to the ecosystem, +this KEP proposed we introduce a subset of `propertyNames` functionality under the +`x-kubernetes-property-names` extension as a way of evolving OpenAPI 3.0 in a direction that is +aligned with OpenAPI 3.1. + +### Goals + +- Enable structural schema validation of map keys (includes `x-kubernetes-validations` support). +- Use `maxLength` on map keys when calculating CEL estimated cost. + +### Non-Goals + +- Implement OpenAPI 3.1. Support for 3.1 is a large, disruptive change. We do not know if/when + this will happen, so continuing to improve OpenAPI 3.0 is pragmatic. + +## Proposal + +- Add `x-kubernetes-property-names` extension. +- Require the use of structural schemas for map keys. +- Update CEL cost estimate system to use `maxLength` of map keys. + +### User Stories (Optional) + +#### Story 1 + +User adds a field to a CRD that should contain only valid Kubernetes labels. + +Solution: + +```yaml +type: object +properties: + labels: + type: object + x-kubernetes-property-names: + type: string + format: k8s-label-key # A proposed format for label keys + additionalProperties: + type: string + format: k8s-label-value # A proposed format for label values +``` + +#### Story 2 + +User introduces a CEL rule that checks map keys, but the CEL estimated cost is too high +if there is no maxLengh limit set on the key field. + +Solution: + +```yaml +type: object +properties: + labels: + type: object + x-kubernetes-property-names: + type: string + maxLength: 64 + additionalProperties: + type: string +``` + +### Notes/Constraints/Caveats (Optional) + +- Map keys MUST be strings. This will be enforced, and only value validations that apply to + strings will be allowed. +- OpenAPI 3.0 libraries that do not recognize the `x-kubernetes-property-names` property are required to ignore it. + This is what makes using an extension preferable to using `propertyNames`. OpenAPI 3.0 libraries may report + an "unrecognized property" error if `propertyNames` is encountered. +- Support for key defaults will also be added. + +### Risks and Mitigations + +Risk: The ecosystem is fragmented by CRD schemas that use this feature, since older versions of Kubernetes +do not support it. + +Mitigation: Ratchet it in. We do this for other capabilities like new formats, CEL functions, ... + +## Design Details + +- Supporting structural schemas makes it easier to reason about what capabilities are available. +- Defaults WILL be supported. + +### Test Plan + +- Add CRD validation tests to test that x-kubernetes-property-names works as expected +- Add CR validation tests covering the new map key validation capabilities +- Add CEL estimated cost tests to ensure estimated costs account for a map key with maxLength + +##### Prerequisite testing updates + + + +##### Unit tests + +- ``: `` - `` + +##### Integration tests + +- [test name](https://github.com/kubernetes/kubernetes/blob/2334b8469e1983c525c0c6382125710093a25883/test/integration/...): [integration master](https://testgrid.k8s.io/sig-release-master-blocking#integration-master?include-filter-by-regex=MyCoolFeature), [triage search](https://storage.googleapis.com/k8s-triage/index.html?test=MyCoolFeature) + +##### e2e tests + + + +- [test name](https://github.com/kubernetes/kubernetes/blob/2334b8469e1983c525c0c6382125710093a25883/test/e2e/...): [SIG ...](https://testgrid.k8s.io/sig-...?include-filter-by-regex=MyCoolFeature), [triage search](https://storage.googleapis.com/k8s-triage/index.html?test=MyCoolFeature) + +### Graduation Criteria + +#### Alpha + +- Feature implemented behind a feature flag +- All unit, integration and e2e tests completed and enabled + + + +### Upgrade / Downgrade Strategy + +During upgrades, apiservers start accepting the new extension field and will start enforcing it. + +During upgrades, apiservers will stop accepting the new extension field and will stop enforcing it. +The field will also be dropped from read requests. + +### Version Skew Strategy + +apiserver versions with the feature enabled will enforce validation while apiserver versions +without the feature will not enforce validation, but will be able to safely read it from storage +and ignore it. + +## Production Readiness Review Questionnaire + + + +### Feature Enablement and Rollback + + + +###### How can this feature be enabled / disabled in a live cluster? + + + +- [x] Feature gate (also fill in values in `kep.yaml`) + - Feature gate name: CRDPropertyNameSchemas + - Components depending on the feature gate: kube-apiserver +- [ ] Other + - Describe the mechanism: + - Will enabling / disabling the feature require downtime of the control + plane? + - Will enabling / disabling the feature require downtime or reprovisioning + of a node? + +###### Does enabling the feature change any default behavior? + +No + +###### Can the feature be disabled once it has been enabled (i.e. can we roll back the enablement)? + +Yes + +###### What happens if we reenable the feature if it was previously rolled back? + +The validation is reactivated. + +###### Are there any tests for feature enablement/disablement? + +Yes, there will be integration tests for this. + +### Rollout, Upgrade and Rollback Planning + + + +###### How can a rollout or rollback fail? Can it impact already running workloads? + + + +###### What specific metrics should inform a rollback? + + + +###### Were upgrade and rollback tested? Was the upgrade->downgrade->upgrade path tested? + + + +###### Is the rollout accompanied by any deprecations and/or removals of features, APIs, fields of API types, flags, etc.? + + + +### Monitoring Requirements + + + +###### How can an operator determine if the feature is in use by workloads? + + + +###### How can someone using this feature know that it is working for their instance? + + + +- [ ] Events + - Event Reason: +- [ ] API .status + - Condition name: + - Other field: +- [ ] Other (treat as last resort) + - Details: + +###### What are the reasonable SLOs (Service Level Objectives) for the enhancement? + + + +###### What are the SLIs (Service Level Indicators) an operator can use to determine the health of the service? + + + +- [ ] Metrics + - Metric name: + - [Optional] Aggregation method: + - Components exposing the metric: +- [ ] Other (treat as last resort) + - Details: + +###### Are there any missing metrics that would be useful to have to improve observability of this feature? + + + +### Dependencies + + + +###### Does this feature depend on any specific services running in the cluster? + + + +### Scalability + + + +###### Will enabling / using this feature result in any new API calls? + + + +###### Will enabling / using this feature result in introducing new API types? + + + +###### Will enabling / using this feature result in any new calls to the cloud provider? + + + +###### Will enabling / using this feature result in increasing size or count of the existing API objects? + + + +###### Will enabling / using this feature result in increasing time taken by any operations covered by existing SLIs/SLOs? + + + +###### Will enabling / using this feature result in non-negligible increase of resource usage (CPU, RAM, disk, IO, ...) in any components? + + + +###### Can enabling / using this feature result in resource exhaustion of some node resources (PIDs, sockets, inodes, etc.)? + + + +### Troubleshooting + + + +###### How does this feature react if the API server and/or etcd is unavailable? + +###### What are other known failure modes? + + + +###### What steps should be taken if SLOs are not being met to determine the problem? + +## Implementation History + + + +## Drawbacks + + + +## Alternatives + + + +## Infrastructure Needed (Optional) + + diff --git a/keps/sig-api-machinery/5667-crd-map-key-validation/kep.yaml b/keps/sig-api-machinery/5667-crd-map-key-validation/kep.yaml new file mode 100644 index 00000000000..bbe2f226ab8 --- /dev/null +++ b/keps/sig-api-machinery/5667-crd-map-key-validation/kep.yaml @@ -0,0 +1,33 @@ +title: CRD Map Key Validation +kep-number: 5667 +authors: + - "@jpbetz" +owning-sig: sig-api-machinery +status: implementable +creation-date: 2025-10-23 +reviewers: + - TBD +approvers: + - "@sttts" + +see-also: + - "/keps/sig-api-machinery/95-custom-resource-definitions" + - "/keps/sig-api-machinery/692-crd-openapi-schema" + - "/keps/sig-api-machinery/2896-openapi-v3" + - "/keps/sig-api-machinery/2876-crd-validation-expression-language" + +stage: alpha + +latest-milestone: "v1.36" + +milestone: + alpha: "v1.36" + +feature-gates: + - name: CRDPropertyNameSchemas + components: + - kube-apiserver +disable-supported: true + +# The following PRR answers are required at beta release +metrics: []