You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: content/en/docs/concepts/extend-kubernetes/api-extension/custom-resources.md
+76-46Lines changed: 76 additions & 46 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -16,8 +16,9 @@ methods for adding custom resources and how to choose between them.
16
16
<!-- body -->
17
17
## Custom resources
18
18
19
-
A *resource* is an endpoint in the [Kubernetes API](/docs/concepts/overview/kubernetes-api/) that stores a collection of
20
-
[API objects](/docs/concepts/overview/working-with-objects/kubernetes-objects/) of a certain kind; for example, the built-in *pods* resource contains a collection of Pod objects.
19
+
A *resource* is an endpoint in the [Kubernetes API](/docs/concepts/overview/kubernetes-api/) that
20
+
stores a collection of [API objects](/docs/concepts/overview/working-with-objects/kubernetes-objects/)
21
+
of a certain kind; for example, the built-in *pods* resource contains a collection of Pod objects.
21
22
22
23
A *custom resource* is an extension of the Kubernetes API that is not necessarily available in a default
23
24
Kubernetes installation. It represents a customization of a particular Kubernetes installation. However,
@@ -68,63 +69,77 @@ or let your API stand alone.
68
69
69
70
In a Declarative API, typically:
70
71
71
-
- Your API consists of a relatively small number of relatively small objects (resources).
72
-
- The objects define configuration of applications or infrastructure.
73
-
- The objects are updated relatively infrequently.
74
-
- Humans often need to read and write the objects.
75
-
- The main operations on the objects are CRUD-y (creating, reading, updating and deleting).
76
-
- Transactions across objects are not required: the API represents a desired state, not an exact state.
72
+
- Your API consists of a relatively small number of relatively small objects (resources).
73
+
- The objects define configuration of applications or infrastructure.
74
+
- The objects are updated relatively infrequently.
75
+
- Humans often need to read and write the objects.
76
+
- The main operations on the objects are CRUD-y (creating, reading, updating and deleting).
77
+
- Transactions across objects are not required: the API represents a desired state, not an exact state.
77
78
78
79
Imperative APIs are not declarative.
79
80
Signs that your API might not be declarative include:
80
81
81
-
- The client says "do this", and then gets a synchronous response back when it is done.
82
-
- The client says "do this", and then gets an operation ID back, and has to check a separate Operation object to determine completion of the request.
83
-
- You talk about Remote Procedure Calls (RPCs).
84
-
- Directly storing large amounts of data; for example, > a few kB per object, or > 1000s of objects.
85
-
- High bandwidth access (10s of requests per second sustained) needed.
86
-
- Store end-user data (such as images, PII, etc.) or other large-scale data processed by applications.
87
-
- The natural operations on the objects are not CRUD-y.
88
-
- The API is not easily modeled as objects.
89
-
- You chose to represent pending operations with an operation ID or an operation object.
82
+
- The client says "do this", and then gets a synchronous response back when it is done.
83
+
- The client says "do this", and then gets an operation ID back, and has to check a separate
84
+
Operation object to determine completion of the request.
85
+
- You talk about Remote Procedure Calls (RPCs).
86
+
- Directly storing large amounts of data; for example, > a few kB per object, or > 1000s of objects.
87
+
- High bandwidth access (10s of requests per second sustained) needed.
88
+
- Store end-user data (such as images, PII, etc.) or other large-scale data processed by applications.
89
+
- The natural operations on the objects are not CRUD-y.
90
+
- The API is not easily modeled as objects.
91
+
- You chose to represent pending operations with an operation ID or an operation object.
90
92
91
93
## Should I use a ConfigMap or a custom resource?
92
94
93
95
Use a ConfigMap if any of the following apply:
94
96
95
-
* There is an existing, well-documented configuration file format, such as a `mysql.cnf` or `pom.xml`.
97
+
* There is an existing, well-documented configuration file format, such as a `mysql.cnf` or
98
+
`pom.xml`.
96
99
* You want to put the entire configuration into one key of a ConfigMap.
97
-
* The main use of the configuration file is for a program running in a Pod on your cluster to consume the file to configure itself.
98
-
* Consumers of the file prefer to consume via file in a Pod or environment variable in a pod, rather than the Kubernetes API.
100
+
* The main use of the configuration file is for a program running in a Pod on your cluster to
101
+
consume the file to configure itself.
102
+
* Consumers of the file prefer to consume via file in a Pod or environment variable in a pod,
103
+
rather than the Kubernetes API.
99
104
* You want to perform rolling updates via Deployment, etc., when the file is updated.
100
105
101
106
{{< note >}}
102
-
Use a {{< glossary_tooltip text="Secret" term_id="secret" >}} for sensitive data, which is similar to a ConfigMap but more secure.
107
+
Use a {{< glossary_tooltip text="Secret" term_id="secret" >}} for sensitive data, which is similar
108
+
to a ConfigMap but more secure.
103
109
{{< /note >}}
104
110
105
111
Use a custom resource (CRD or Aggregated API) if most of the following apply:
106
112
107
113
* You want to use Kubernetes client libraries and CLIs to create and update the new resource.
108
114
* You want top-level support from `kubectl`; for example, `kubectl get my-object object-name`.
109
-
* You want to build new automation that watches for updates on the new object, and then CRUD other objects, or vice versa.
115
+
* You want to build new automation that watches for updates on the new object, and then CRUD other
116
+
objects, or vice versa.
110
117
* You want to write automation that handles updates to the object.
111
118
* You want to use Kubernetes API conventions like `.spec`, `.status`, and `.metadata`.
112
-
* You want the object to be an abstraction over a collection of controlled resources, or a summarization of other resources.
119
+
* You want the object to be an abstraction over a collection of controlled resources, or a
120
+
summarization of other resources.
113
121
114
122
## Adding custom resources
115
123
116
124
Kubernetes provides two ways to add custom resources to your cluster:
117
125
118
126
- CRDs are simple and can be created without any programming.
119
-
-[API Aggregation](/docs/concepts/extend-kubernetes/api-extension/apiserver-aggregation/) requires programming, but allows more control over API behaviors like how data is stored and conversion between API versions.
requires programming, but allows more control over API behaviors like how data is stored and
129
+
conversion between API versions.
120
130
121
-
Kubernetes provides these two options to meet the needs of different users, so that neither ease of use nor flexibility is compromised.
131
+
Kubernetes provides these two options to meet the needs of different users, so that neither ease
132
+
of use nor flexibility is compromised.
122
133
123
-
Aggregated APIs are subordinate API servers that sit behind the primary API server, which acts as a proxy. This arrangement is called [API Aggregation](/docs/concepts/extend-kubernetes/api-extension/apiserver-aggregation/) (AA). To users, the Kubernetes API appears extended.
134
+
Aggregated APIs are subordinate API servers that sit behind the primary API server, which acts as
135
+
a proxy. This arrangement is called [API Aggregation](/docs/concepts/extend-kubernetes/api-extension/apiserver-aggregation/)(AA).
136
+
To users, the Kubernetes API appears extended.
124
137
125
-
CRDs allow users to create new types of resources without adding another API server. You do not need to understand API Aggregation to use CRDs.
138
+
CRDs allow users to create new types of resources without adding another API server. You do not
139
+
need to understand API Aggregation to use CRDs.
126
140
127
-
Regardless of how they are installed, the new resources are referred to as Custom Resources to distinguish them from built-in Kubernetes resources (like pods).
141
+
Regardless of how they are installed, the new resources are referred to as Custom Resources to
142
+
distinguish them from built-in Kubernetes resources (like pods).
128
143
129
144
{{< note >}}
130
145
Avoid using a Custom Resource as data storage for application, end user, or monitoring data:
@@ -156,10 +171,14 @@ and use a controller to handle events.
156
171
157
172
## API server aggregation
158
173
159
-
Usually, each resource in the Kubernetes API requires code that handles REST requests and manages persistent storage of objects. The main Kubernetes API server handles built-in resources like *pods* and *services*, and can also generically handle custom resources through [CRDs](#customresourcedefinitions).
174
+
Usually, each resource in the Kubernetes API requires code that handles REST requests and manages
175
+
persistent storage of objects. The main Kubernetes API server handles built-in resources like
176
+
*pods* and *services*, and can also generically handle custom resources through
177
+
[CRDs](#customresourcedefinitions).
160
178
161
-
The [aggregation layer](/docs/concepts/extend-kubernetes/api-extension/apiserver-aggregation/) allows you to provide specialized
162
-
implementations for your custom resources by writing and deploying your own API server.
179
+
The [aggregation layer](/docs/concepts/extend-kubernetes/api-extension/apiserver-aggregation/)
180
+
allows you to provide specialized implementations for your custom resources by writing and
181
+
deploying your own API server.
163
182
The main API server delegates requests to your API server for the custom resources that you handle,
164
183
making them available to all of its clients.
165
184
@@ -170,7 +189,8 @@ CRDs are easier to use. Aggregated APIs are more flexible. Choose the method tha
170
189
Typically, CRDs are a good fit if:
171
190
172
191
* You have a handful of fields
173
-
* You are using the resource within your company, or as part of a small open-source project (as opposed to a commercial product)
192
+
* You are using the resource within your company, or as part of a small open-source project (as
193
+
opposed to a commercial product)
174
194
175
195
### Comparing ease of use
176
196
@@ -203,7 +223,8 @@ Aggregated APIs offer more advanced API features and customization of other feat
203
223
204
224
### Common Features
205
225
206
-
When you create a custom resource, either via a CRD or an AA, you get many features for your API, compared to implementing it outside the Kubernetes platform:
226
+
When you create a custom resource, either via a CRD or an AA, you get many features for your API,
227
+
compared to implementing it outside the Kubernetes platform:
207
228
208
229
| Feature | What it does |
209
230
| ------- | ------------ |
@@ -228,42 +249,51 @@ There are several points to be aware of before adding a custom resource to your
228
249
229
250
### Third party code and new points of failure
230
251
231
-
While creating a CRD does not automatically add any new points of failure (for example, by causing third party code to run on your API server), packages (for example, Charts) or other installation bundles often include CRDs as well as a Deployment of third-party code that implements the business logic for a new custom resource.
252
+
While creating a CRD does not automatically add any new points of failure (for example, by causing
253
+
third party code to run on your API server), packages (for example, Charts) or other installation
254
+
bundles often include CRDs as well as a Deployment of third-party code that implements the
255
+
business logic for a new custom resource.
232
256
233
257
Installing an Aggregated API server always involves running a new Deployment.
234
258
235
259
### Storage
236
260
237
-
Custom resources consume storage space in the same way that ConfigMaps do. Creating too many custom resources may overload your API server's storage space.
261
+
Custom resources consume storage space in the same way that ConfigMaps do. Creating too many
262
+
custom resources may overload your API server's storage space.
238
263
239
-
Aggregated API servers may use the same storage as the main API server, in which case the same warning applies.
264
+
Aggregated API servers may use the same storage as the main API server, in which case the same
265
+
warning applies.
240
266
241
267
### Authentication, authorization, and auditing
242
268
243
-
CRDs always use the same authentication, authorization, and audit logging as the built-in resources of your API server.
269
+
CRDs always use the same authentication, authorization, and audit logging as the built-in
270
+
resources of your API server.
244
271
245
-
If you use RBAC for authorization, most RBAC roles will not grant access to the new resources (except the cluster-admin role or any role created with wildcard rules). You'll need to explicitly grant access to the new resources. CRDs and Aggregated APIs often come bundled with new role definitions for the types they add.
272
+
If you use RBAC for authorization, most RBAC roles will not grant access to the new resources
273
+
(except the cluster-admin role or any role created with wildcard rules). You'll need to explicitly
274
+
grant access to the new resources. CRDs and Aggregated APIs often come bundled with new role
275
+
definitions for the types they add.
246
276
247
-
Aggregated API servers may or may not use the same authentication, authorization, and auditing as the primary API server.
277
+
Aggregated API servers may or may not use the same authentication, authorization, and auditing as
278
+
the primary API server.
248
279
249
280
## Accessing a custom resource
250
281
251
-
Kubernetes [client libraries](/docs/reference/using-api/client-libraries/) can be used to access custom resources. Not all client libraries support custom resources. The _Go_ and _Python_ client libraries do.
282
+
Kubernetes [client libraries](/docs/reference/using-api/client-libraries/) can be used to access
283
+
custom resources. Not all client libraries support custom resources. The _Go_ and _Python_ client
284
+
libraries do.
252
285
253
286
When you add a custom resource, you can access it using:
254
287
255
288
-`kubectl`
256
289
- The Kubernetes dynamic client.
257
290
- A REST client that you write.
258
-
- A client generated using [Kubernetes client generation tools](https://github.com/kubernetes/code-generator) (generating one is an advanced undertaking, but some projects may provide a client along with the CRD or AA).
259
-
260
-
291
+
- A client generated using [Kubernetes client generation tools](https://github.com/kubernetes/code-generator)
292
+
(generating one is an advanced undertaking, but some projects may provide a client along with
293
+
the CRD or AA).
261
294
262
295
## {{% heading "whatsnext" %}}
263
296
264
-
265
297
* Learn how to [Extend the Kubernetes API with the aggregation layer](/docs/concepts/extend-kubernetes/api-extension/apiserver-aggregation/).
266
-
267
298
* Learn how to [Extend the Kubernetes API with CustomResourceDefinition](/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/).
0 commit comments