Skip to content

Commit 9fe0b4c

Browse files
liggittliggitt
committed
Describe process for allowing a new enum value in an existing field
1 parent 39d25a2 commit 9fe0b4c

File tree

1 file changed

+162
-3
lines changed

1 file changed

+162
-3
lines changed

‎contributors/devel/sig-architecture/api_changes.md

Lines changed: 162 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -19,8 +19,12 @@ found at [API Conventions](api-conventions.md).
1919
- [Edit types.go](#edit-typesgo-1)
2020
- [Edit validation.go](#edit-validationgo)
2121
- [Edit version conversions](#edit-version-conversions)
22-
- [Generate protobuf objects](#generate-protobuf-objects)
23-
- [Edit json (un)marshaling code](#edit-json-unmarshaling-code)
22+
- [Generate Code](#generate-code)
23+
- [Generate protobuf objects](#generate-protobuf-objects)
24+
- [Generate Clientset](#generate-clientset)
25+
- [Generate Listers](#generate-listers)
26+
- [Generate Informers](#generate-informers)
27+
- [Edit json (un)marshaling code](#edit-json-unmarshaling-code)
2428
- [Making a new API Version](#making-a-new-api-version)
2529
- [Making a new API Group](#making-a-new-api-group)
2630
- [Update the fuzzer](#update-the-fuzzer)
@@ -30,6 +34,9 @@ found at [API Conventions](api-conventions.md).
3034
- [Examples and docs](#examples-and-docs)
3135
- [Alpha, Beta, and Stable Versions](#alpha-beta-and-stable-versions)
3236
- [Adding Unstable Features to Stable Versions](#adding-unstable-features-to-stable-versions)
37+
- [New field in existing API version](#new-field-in-existing-api-version)
38+
- [New enum value in existing field](#new-enum-value-in-existing-field)
39+
- [New alpha API version](#new-alpha-api-version)
3340

3441
## So you want to change the API?
3542

@@ -880,7 +887,10 @@ users are only able or willing to accept a released version of Kubernetes. In
880887
that case, the developer has a few options, both of which require staging work
881888
over several releases.
882889

883-
#### Alpha field in existing API version
890+
The mechanism used depends on whether a new field is being added,
891+
or a new value is being permitted in an existing field.
892+
893+
#### New field in existing API version
884894

885895
Previously, annotations were used for experimental alpha features, but are no longer recommended for several reasons:
886896

@@ -1016,6 +1026,155 @@ In future Kubernetes versions:
10161026
}
10171027
```
10181028

1029+
#### New enum value in existing field
1030+
1031+
A developer is considering adding a new allowed enum value of `"OnlyOnTuesday"`
1032+
to the following existing enum field:
1033+
1034+
```go
1035+
type Frobber struct {
1036+
// restartPolicy may be set to "Always" or "Never".
1037+
// Additional policies may be defined in the future.
1038+
// Unrecognized policies should be treated as "Never".
1039+
RestartPolicy string `json:"policy"
1040+
}
1041+
```
1042+
1043+
Older versions of expected API clients must be able handle the new value in a safe way:
1044+
1045+
* If the enum field drives behavior of a single component, ensure all versions of that component
1046+
that will encounter API objects containing the new value handle it properly or fail safe.
1047+
For example, a new allowed value in a `Pod` enum field consumed by the kubelet must be handled
1048+
safely by kubelets up to two versions older than the first API server release that allowed the new value.
1049+
* If an API drives behavior that is implemented by external clients (like `Ingress` or `NetworkPolicy`),
1050+
the enum field must explicitly indicate that additional values may be allowed in the future,
1051+
and define how unrecognized values must be handled by clients. If this was not done in the first release
1052+
containing the enum field, it is not safe to add new values that can break existing clients.
1053+
1054+
If expected API clients safely handle the new enum value, the next requirement is to begin allowing it
1055+
in a way that does not break validation of that object by a previous API server.
1056+
This requires at least two releases to accomplish safely:
1057+
1058+
Release 1:
1059+
1060+
* Only allow the new enum value when updating existing objects that already contain the new enum value
1061+
* Disallow it in other cases (creation, and update of objects that do not already contain the new enum value)
1062+
1063+
Release 2:
1064+
1065+
* Allow the new enum value in create and update scenarios
1066+
1067+
This ensures a cluster with multiple servers at skewed releases (which happens during a rolling upgrade),
1068+
will not allow data to be persisted which the previous release of the API server would choke on.
1069+
1070+
Typically, a feature gate is used to do this rollout, starting in alpha and disabled by default in release 1,
1071+
and graduating to beta and enabled by default in release 2.
1072+
1073+
1. Add a feature gate to the API server to control enablement of the new enum value (and associated function):
1074+
1075+
In [staging/src/k8s.io/apiserver/pkg/features/kube_features.go](https://git.k8s.io/kubernetes/staging/src/k8s.io/apiserver/pkg/features/kube_features.go):
1076+
1077+
```go
1078+
// owner: @you
1079+
// alpha: v1.11
1080+
//
1081+
// Allow OnTuesday restart policy in frobbers.
1082+
FrobberRestartPolicyOnTuesday utilfeature.Feature = "FrobberRestartPolicyOnTuesday"
1083+
1084+
var defaultKubernetesFeatureGates = map[utilfeature.Feature]utilfeature.FeatureSpec{
1085+
...
1086+
FrobberRestartPolicyOnTuesday: {Default: false, PreRelease: utilfeature.Alpha},
1087+
}
1088+
```
1089+
1090+
2. Update the documentation on the API type:
1091+
1092+
* include details about the alpha-level in the field description
1093+
1094+
```go
1095+
type Frobber struct {
1096+
// restartPolicy may be set to "Always" or "Never" (or "OnTuesday" if the alpha "FrobberRestartPolicyOnTuesday" feature is enabled).
1097+
// Additional policies may be defined in the future.
1098+
// Unrecognized policies should be treated as "Never".
1099+
RestartPolicy string `json:"policy"
1100+
}
1101+
```
1102+
1103+
3. When validating the object, determine whether the new enum value should be allowed.
1104+
This prevents new usage of the new value when the feature is disabled, while ensuring existing data is preserved.
1105+
Ensuring existing data is preserved is needed so that when the feature is enabled by default in a future version *n*
1106+
and data is unconditionally allowed to be persisted in the field, an *n-1* API server
1107+
(with the feature still disabled by default) will not choke on validation.
1108+
The recommended place to do this is in the REST storage strategy's Validate/ValidateUpdate methods:
1109+
1110+
```go
1111+
func (frobberStrategy) Validate(ctx genericapirequest.Context, obj runtime.Object) field.ErrorList {
1112+
frobber := obj.(*api.Frobber)
1113+
return validation.ValidateFrobber(frobber, validationOptionsForFrobber(frobber, nil))
1114+
}
1115+
1116+
func (frobberStrategy) ValidateUpdate(ctx genericapirequest.Context, obj, old runtime.Object) field.ErrorList {
1117+
newFrobber := obj.(*api.Frobber)
1118+
oldFrobber := old.(*api.Frobber)
1119+
return validation.ValidateFrobberUpdate(newFrobber, oldFrobber, validationOptionsForFrobber(newFrobber, oldFrobber))
1120+
}
1121+
1122+
func validationOptionsForFrobber(newFrobber, oldFrobber *api.Frobber) validation.FrobberValidationOptions {
1123+
opts := validation.FrobberValidationOptions{
1124+
// allow if the feature is enabled
1125+
AllowRestartPolicyOnTuesday: utilfeature.DefaultFeatureGate.Enabled(features.Frobber2D)
1126+
}
1127+
1128+
if oldFrobber == nil {
1129+
// if there's no old object, use the options based solely on feature enablement
1130+
return opts
1131+
}
1132+
1133+
if oldFrobber.RestartPolicy == api.RestartPolicyOnTuesday {
1134+
// if the old object already used the enum value, continue to allow it in the new object
1135+
opts.AllowRestartPolicyOnTuesday = true
1136+
}
1137+
return opts
1138+
}
1139+
```
1140+
1141+
4. In validation, validate the enum value based on the passed-in options:
1142+
1143+
```go
1144+
func ValidateFrobber(f *api.Frobber, opts FrobberValidationOptions) field.ErrorList {
1145+
...
1146+
validRestartPolicies := sets.NewString(RestartPolicyAlways, RestartPolicyNever)
1147+
if opts.AllowRestartPolicyOnTuesday {
1148+
validRestartPolicies.Insert(RestartPolicyOnTuesday)
1149+
}
1150+
1151+
if f.RestartPolicy == RestartPolicyOnTuesday && !opts.AllowRestartPolicyOnTuesday {
1152+
allErrs = append(allErrs, field.Invalid(field.NewPath("restartPolicy"), f.RestartPolicy, "only allowed if the FrobberRestartPolicyOnTuesday feature is enabled"))
1153+
} else if !validRestartPolicies.Has(f.RestartPolicy) {
1154+
allErrs = append(allErrs, field.NotSupported(field.NewPath("restartPolicy"), f.RestartPolicy, validRestartPolicies.List()))
1155+
}
1156+
...
1157+
}
1158+
```
1159+
1160+
5. After at least one release, the feature can be promoted to beta or GA and enabled by default.
1161+
1162+
In [staging/src/k8s.io/apiserver/pkg/features/kube_features.go](https://git.k8s.io/kubernetes/staging/src/k8s.io/apiserver/pkg/features/kube_features.go):
1163+
1164+
```go
1165+
// owner: @you
1166+
// alpha: v1.11
1167+
// beta: v1.12
1168+
//
1169+
// Allow OnTuesday restart policy in frobbers.
1170+
FrobberRestartPolicyOnTuesday utilfeature.Feature = "FrobberRestartPolicyOnTuesday"
1171+
1172+
var defaultKubernetesFeatureGates = map[utilfeature.Feature]utilfeature.FeatureSpec{
1173+
...
1174+
FrobberRestartPolicyOnTuesday: {Default: true, PreRelease: utilfeature.Beta},
1175+
}
1176+
```
1177+
10191178
#### New alpha API version
10201179

10211180
Another option is to introduce a new type with an new `alpha` or `beta` version

0 commit comments

Comments
 (0)