Skip to content

Commit 47da91b

Browse files
authored
Merge pull request #6764 from thockin/master
API conventions: add more on defaulting
2 parents 5d1e5d6 + b7bd632 commit 47da91b

File tree

1 file changed

+220
-64
lines changed

1 file changed

+220
-64
lines changed

contributors/devel/sig-architecture/api-conventions.md

Lines changed: 220 additions & 64 deletions
Original file line numberDiff line numberDiff line change
@@ -7,40 +7,66 @@ An introduction to using resources with kubectl can be found in [the object mana
77

88
**Table of Contents**
99

10-
11-
- [Types (Kinds)](#types-kinds)
12-
- [Resources](#resources)
13-
- [Objects](#objects)
14-
- [Metadata](#metadata)
15-
- [Spec and Status](#spec-and-status)
16-
- [Typical status properties](#typical-status-properties)
17-
- [References to related objects](#references-to-related-objects)
18-
- [Lists of named subobjects preferred over maps](#lists-of-named-subobjects-preferred-over-maps)
19-
- [Primitive types](#primitive-types)
20-
- [Constants](#constants)
21-
- [Unions](#unions)
22-
- [Lists and Simple kinds](#lists-and-simple-kinds)
23-
- [Differing Representations](#differing-representations)
24-
- [Verbs on Resources](#verbs-on-resources)
25-
- [PATCH operations](#patch-operations)
26-
- [Idempotency](#idempotency)
27-
- [Optional vs. Required](#optional-vs-required)
28-
- [Defaulting](#defaulting)
29-
- [Late Initialization](#late-initialization)
30-
- [Concurrency Control and Consistency](#concurrency-control-and-consistency)
31-
- [Serialization Format](#serialization-format)
32-
- [Units](#units)
33-
- [Selecting Fields](#selecting-fields)
34-
- [Object references](#object-references)
35-
- [HTTP Status codes](#http-status-codes)
36-
- [Success codes](#success-codes)
37-
- [Error codes](#error-codes)
38-
- [Response Status Kind](#response-status-kind)
39-
- [Events](#events)
40-
- [Naming conventions](#naming-conventions)
41-
- [Label, selector, and annotation conventions](#label-selector-and-annotation-conventions)
42-
- [WebSockets and SPDY](#websockets-and-spdy)
43-
- [Validation](#validation)
10+
- [Types (Kinds)](#types-kinds)
11+
- [Resources](#resources)
12+
- [Objects](#objects)
13+
- [Metadata](#metadata)
14+
- [Spec and Status](#spec-and-status)
15+
- [Typical status properties](#typical-status-properties)
16+
- [References to related objects](#references-to-related-objects)
17+
- [Lists of named subobjects preferred over maps](#lists-of-named-subobjects-preferred-over-maps)
18+
- [Primitive types](#primitive-types)
19+
- [Constants](#constants)
20+
- [Unions](#unions)
21+
- [Lists and Simple kinds](#lists-and-simple-kinds)
22+
- [Differing Representations](#differing-representations)
23+
- [Verbs on Resources](#verbs-on-resources)
24+
- [PATCH operations](#patch-operations)
25+
- [Idempotency](#idempotency)
26+
- [Optional vs. Required](#optional-vs-required)
27+
- [Defaulting](#defaulting)
28+
- [Static Defaults](#static-defaults)
29+
- [Admission Controlled Defaults](#admission-controlled-defaults)
30+
- [Controller-Assigned Defaults (aka Late Initialization)](#controller-assigned-defaults-aka-late-initialization)
31+
- [What May Be Defaulted](#what-may-be-defaulted)
32+
- [Considerations For PUT Operations](#considerations-for-put-operations)
33+
- [Concurrency Control and Consistency](#concurrency-control-and-consistency)
34+
- [Serialization Format](#serialization-format)
35+
- [Units](#units)
36+
- [Selecting Fields](#selecting-fields)
37+
- [Object references](#object-references)
38+
- [Naming of the reference field](#naming-of-the-reference-field)
39+
- [Referencing resources with multiple versions](#referencing-resources-with-multiple-versions)
40+
- [Handling of resources that do not exist](#handling-of-resources-that-do-not-exist)
41+
- [Validation of fields](#validation-of-fields)
42+
- [Do not modify the referred object](#do-not-modify-the-referred-object)
43+
- [Minimize copying or printing values to the referrer object](#minimize-copying-or-printing-values-to-the-referrer-object)
44+
- [Object References Examples](#object-references-examples)
45+
- [Single resource reference](#single-resource-reference)
46+
- [Controller behavior](#controller-behavior)
47+
- [Multiple resource reference](#multiple-resource-reference)
48+
- [Kind vs. Resource](#kind-vs-resource)
49+
- [Controller behavior](#controller-behavior-1)
50+
- [Generic object reference](#generic-object-reference)
51+
- [Controller behavior](#controller-behavior-2)
52+
- [Field reference](#field-reference)
53+
- [Controller behavior](#controller-behavior-3)
54+
- [HTTP Status codes](#http-status-codes)
55+
- [Success codes](#success-codes)
56+
- [Error codes](#error-codes)
57+
- [Response Status Kind](#response-status-kind)
58+
- [Events](#events)
59+
- [Naming conventions](#naming-conventions)
60+
- [Namespace Names](#namespace-names)
61+
- [Label, selector, and annotation conventions](#label-selector-and-annotation-conventions)
62+
- [WebSockets and SPDY](#websockets-and-spdy)
63+
- [Validation](#validation)
64+
- [Automatic Resource Allocation And Deallocation](#automatic-resource-allocation-and-deallocation)
65+
- [Representing Allocated Values](#representing-allocated-values)
66+
- [When to use a <code>spec</code> field](#when-to-use-a-spec-field)
67+
- [When to use a <code>status</code> field](#when-to-use-a-status-field)
68+
- [Sequencing operations](#sequencing-operations)
69+
- [When to use a different type](#when-to-use-a-different-type)
4470

4571

4672
The conventions of the [Kubernetes API](https://kubernetes.io/docs/api/) (and related APIs in the
@@ -742,51 +768,181 @@ have a built-in `nil` value.
742768

743769
## Defaulting
744770

745-
Default resource values are API version-specific, and they are applied during
746-
the conversion from API-versioned declarative configuration to internal objects
747-
representing the desired state (`Spec`) of the resource. Subsequent GETs of the
748-
resource will include the default values explicitly.
749-
750-
Incorporating the default values into the `Spec` ensures that `Spec` depicts the
751-
full desired state so that it is easier for the system to determine how to
752-
achieve the state, and for the user to know what to anticipate.
771+
In general we want default values to be explicitly represented in our APIs,
772+
rather than asserting that "unspecified fields get the default behavior". This
773+
is important so that:
774+
- default values can evolve and change in newer API versions
775+
- the stored configuration depicts the full desired state, making it easier
776+
for the system to determine how to achieve the state, and for the user to
777+
know what to anticipate
778+
779+
There are 3 distinct ways that default values can be applied when creating or
780+
updating (including patch and apply) a resource:
781+
782+
1. static: based on the requested API version and possibly other fields in the
783+
resource, fields can be assigned values during the API call
784+
2. admission control: based on the configured admission controllers and
785+
possibly other state in or out of the cluster, fields can be assigned
786+
values during the API call
787+
3. controllers: arbitrary changes (within the bounds of what is allowed) can
788+
be made to a resource after the API call has completed
789+
790+
Some care is required when deciding which mechanism to use and managing the
791+
semantics.
792+
793+
### Static Defaults
794+
795+
Static default values are specific to each API version. The default field
796+
values applied when creating an object with the "v1" API may be different than
797+
the values applied when using the "v2" API. In most cases, these values are
798+
defined as literal values by the API version (e.g. "if this field is not
799+
specified it defaults to 0").
800+
801+
In some cases, these values may be conditional on or deterministically derived
802+
from other fields (e.g. "if otherField is X then this field defaults to 0" or
803+
"this field defaults to the value of otherField"). Note that such derived
804+
defaults present a hazard in the face of updates - if the "other" field
805+
changes, the derived field may have to change, too. The static defaulting
806+
logic is unaware of updates and has no concept of "previous value", which means
807+
this inter-field relationship becomes the user's problem - they must update
808+
both the field they care about and the "other" field.
809+
810+
In very rare cases, these values may be allocated from some pool or determined
811+
by some other method (e.g. Service's IP and IP-family related fields need to
812+
consider other configuration settings).
813+
814+
These values are applied synchronously by the API server when decoding
815+
versioned data. For CREATE and UPDATE operations this is fairly
816+
straight-forward - when the API server receives a (versioned) request, the
817+
default values are immediately applied before any further processing. When the
818+
API call completes, all static defaults will have been set and stored.
819+
Subsequent GETs of the resource will include the default values explicitly.
820+
However, static defaults also apply when an object is read from storage (i.e.
821+
GET operations). This means that when someone GETs an "older" stored object,
822+
any fields which have been added to the API since that object was stored will
823+
be defaulted and returned according to the API version that is stored.
824+
825+
Static defaults are the best choice for values which are logically required,
826+
but which have a value that works well for most users. Static defaulting
827+
must not consider any state except the object being operated upon (and the
828+
complexity of Service API stands as an example of why).
753829

754830
Default values can be specified on a field using the `+default=` tag. Primitives
755831
will have their values directly assigned while structs will go through the
756832
JSON unmarshalling process. Fields that do not have an `omitempty` json tag will
757833
default to the zero value of their corresponding type if no default is assigned.
758834

759-
Refer to [defaulting docs](https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/#defaulting) for more information.
835+
Refer to [defaulting docs](https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/#defaulting)
836+
for more information.
837+
838+
### Admission Controlled Defaults
839+
840+
In some cases, it is useful to set a default value which is not derived from
841+
the object in question. For example, when creating a PersistentVolumeClaim,
842+
the storage class must be specified. For many users, the best answer is
843+
"whatever the cluster admin has decided for the default". StorageClass is a
844+
different API than PersistentVolumeClaim, and which one is denoted as the
845+
default may change at any time. Thus this is not eligible for static
846+
defaulting.
760847

761-
API version-specific default values are set by the API server.
848+
Instead, we can provide a built-in admission controller or a
849+
MutatingWebhookConfiguration. Unlike static defaults, these may consider
850+
external state (such as annotations on StorageClass objects) when deciding
851+
default values, and must handle things like race conditions (e.g. a
852+
StorageClass is designated the default, but the admission controller has not
853+
yet seen that update). These admission controllers are strictly optional and
854+
can be disabled. As such, fields which are initialized this way must be
855+
strictly optional.
762856

763-
## Late Initialization
857+
Like static defaults, these are run synchronously to the API operation in
858+
question, and when the API call completes, all static defaults will have been
859+
set. Subsequent GETs of the resource will include the default values
860+
explicitly.
861+
862+
### Controller-Assigned Defaults (aka Late Initialization)
764863

765864
Late initialization is when resource fields are set by a system controller
766-
after an object is created/updated.
865+
after an object is created/updated (asynchronously). For example, the
866+
scheduler sets the `pod.spec.nodeName` field after the pod is created. It's
867+
a stretch to call this "defaulting" but since it is so common and useful, it is
868+
included here.
869+
870+
Like admission controlled defaults, these controllers may consider external
871+
state when deciding what values to set, must handle race conditions, and can be
872+
disabled. Fields which are initialized this way must be strictly optional
873+
(meaning observers will see the object without these fields set, and that is
874+
allowable and semantically correct).
875+
876+
Like all controllers, care must be taken to not clobber unrelated fields or
877+
values (e.g. in an array). Using one of the patch or apply mechanisms is
878+
recommended to facilitate composition and concurrency of controllers.
767879

768-
For example, the scheduler sets the `pod.spec.nodeName` field after the pod is
769-
created.
880+
### What May Be Defaulted
770881

771-
Late-initializers should only make the following types of modifications:
882+
All forms of defaulting should only make the following types of modifications:
772883
- Setting previously unset fields
773884
- Adding keys to maps
774885
- Adding values to arrays which have mergeable semantics
775-
(`patchStrategy:"merge"` attribute in the type definition).
776-
777-
These conventions:
778-
1. allow a user (with sufficient privilege) to override any system-default
779-
behaviors by setting the fields that would otherwise have been defaulted.
780-
1. enables updates from users to be merged with changes made during late
781-
initialization, using strategic merge patch, as opposed to clobbering the
782-
change.
783-
1. allow the component which does the late-initialization to use strategic
784-
merge patch, which facilitates composition and concurrency of such components.
785-
786-
Although the apiserver Admission Control stage acts prior to object creation,
787-
Admission Control plugins should follow the Late Initialization conventions
788-
too, to allow their implementation to be later moved to a 'controller', or to
789-
client libraries.
886+
(`patchStrategy:"merge"` attribute in the type definition)
887+
888+
In particular we never want to change or override a value that was provided by
889+
the user. If they requested something invalid, they should get an error.
890+
891+
These rules ensure that:
892+
1. a user (with sufficient privilege) can override any system-default
893+
behaviors by explicitly setting the fields that would otherwise have been
894+
defaulted
895+
1. updates from users can be merged with default values
896+
897+
### Considerations For PUT Operations
898+
899+
Once an object has been created and defaults have been applied, it's very
900+
common for updates to happen over time. Kubernetes offers several ways of
901+
updating an object which preserve existing values in fields other than those
902+
being updated (e.g. strategic merge patch and server-side apply). There is,
903+
however, a less obvious way of updating objects which can have bad interactions
904+
with default values - PUT (aka `kubectl replace`).
905+
906+
The goal is that, for a given input (e.g. YAML file), PUT on an existing object
907+
should produce the same result as if you used that input to create the object.
908+
Calling PUT a second time with the same input should be idempotent and should
909+
not change the resource. Even a read-modify-write cycle is not a perfect
910+
solution in the face of version skew.
911+
912+
When an object is updated with a PUT, the API server will see the "old" object
913+
with previously assigned defaults and the "new" object with newly assigned
914+
defaults. For static defaults this can be a problem if the CREATE and the PUT
915+
used different API versions. For example, "v1" of an API might default a field
916+
to `false`, while "v2" defaults it to `true`. If an object was created via API
917+
v1 (field = `false`) and then replaced via API v2, the field will attempt to
918+
change to `true`. This can also be a problem when the values are allocated or
919+
derived from a source outside of the object in question (e.g. Service IPs).
920+
921+
For some APIs this is acceptable and actionable. For others, this may be
922+
disallowed by validation. In the latter case, the user will get an error about
923+
an attempt to change a field which is not even present in their YAML. This is
924+
especially dangerous when adding new fields - an older client may not even know
925+
about the existence of the field, making even a read-modify-write cycle fail.
926+
While it is "correct" (in the sense that it is really what they asked for with
927+
PUT), it is not helpful and is a bad UX.
928+
929+
When adding a field with a static or admission controlled default, this must be
930+
considered. If the field is immutable after creation, consider adding logic to
931+
manually "patch" the value from the "old" object into the "new" one when it has
932+
been "unset", rather than returning an error or allocating a different value
933+
(e.g. Service IPs). This will very often be what the user meant, even if it
934+
is not what they said. This may require setting the default in a different way
935+
(e.g. in the registry code which understands updates instead of in the
936+
versioned defaulting code which does not). Be careful to detect and report
937+
legitimate errors where the "new" value is specified but is different from the
938+
"old" value.
939+
940+
For controller-defaulted fields, the situation is even more unpleasant.
941+
Controllers do not have an opportunity to "patch" the value before the API
942+
operation is committed. If the "unset" value is allowed then it will be saved,
943+
and any watch clients will be notified. If the "unset" value is not allowed or
944+
mutations are otherwise disallowed, the user will get an error, and there's
945+
simply nothing we can do about it.
790946

791947
## Concurrency Control and Consistency
792948

0 commit comments

Comments
 (0)