Skip to content

Commit 9a804d2

Browse files
k8s-ci-robotk8s-ci-robot
authored
Merge pull request #39642 from jpbetz/cel-docs
Add reference page for CEL in Kubernetes
2 parents c05bf92 + e28b74f commit 9a804d2

File tree

1 file changed

+293
-0
lines changed
  • content/en/docs/reference/using-api

1 file changed

+293
-0
lines changed

‎content/en/docs/reference/using-api/cel.md

Lines changed: 293 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,293 @@
1+
---
2+
title: Common Expression Language in Kubernetes
3+
reviewers:
4+
- jpbetz
5+
- cici37
6+
content_type: concept
7+
weight: 35
8+
min-kubernetes-server-version: 1.25
9+
---
10+
11+
<!-- overview -->
12+
13+
The [Common Expression Language (CEL)](https://github.com/google/cel-go) is used
14+
in the Kubernetes API to declare validation rules, policy rules, and other
15+
constraints or conditions.
16+
17+
CEL expressions are evaluated directly in the
18+
{{< glossary_tooltip text="API server" term_id="kube-apiserver" >}}, making CEL a
19+
convenient alternative to out-of-process mechanisms, such as webhooks, for many
20+
extensibility use cases. Your CEL expressions continue to execute so long as the
21+
control plane's API server component remains available.
22+
23+
<!-- body -->
24+
25+
## Language overview
26+
27+
The [CEL
28+
language](https://github.com/google/cel-spec/blob/master/doc/langdef.md) has a
29+
straightforward syntax that is similar to the expressions in C, C++, Java,
30+
JavaScript and Go.
31+
32+
CEL was designed to be embedded into applications. Each CEL "program" is a
33+
single expression that evaluates to a single value. CEL expressions are
34+
typically short "one-liners" that inline well into the string fields of Kubernetes
35+
API resources.
36+
37+
Inputs to a CEL program are "variables". Each Kubernetes API field that contains
38+
CEL declares in the API documentation which variables are available to use for
39+
that field. For example, in the `x-kubernetes-validations[i].rules` field of
40+
CustomResourceDefinitions, the `self` and `oldSelf` variables are available and
41+
refer to the previous and current state of the custom resource data to be
42+
validated by the CEL expression. Other Kubernetes API fields may declare
43+
different variables. See the API documentation of the API fields to learn which
44+
variables are available for that field.
45+
46+
Example CEL expressions:
47+
48+
{{< table caption="Examples of CEL expressions and the purpose of each" >}}
49+
| Rule | Purpose |
50+
|------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------|
51+
| `self.minReplicas <= self.replicas && self.replicas <= self.maxReplicas` | Validate that the three fields defining replicas are ordered appropriately |
52+
| `'Available' in self.stateCounts` | Validate that an entry with the 'Available' key exists in a map |
53+
| `(self.list1.size() == 0) != (self.list2.size() == 0)` | Validate that one of two lists is non-empty, but not both |
54+
| `self.envars.filter(e, e.name = 'MY_ENV').all(e, e.value.matches('^[a-zA-Z]*$')` | Validate the 'value' field of a listMap entry where key field 'name' is 'MY_ENV' |
55+
| `has(self.expired) && self.created + self.ttl < self.expired` | Validate that 'expired' date is after a 'create' date plus a 'ttl' duration |
56+
| `self.health.startsWith('ok')` | Validate a 'health' string field has the prefix 'ok' |
57+
| `self.widgets.exists(w, w.key == 'x' && w.foo < 10)` | Validate that the 'foo' property of a listMap item with a key 'x' is less than 10 |
58+
| `type(self) == string ? self == '99%' : self == 42` | Validate an int-or-string field for both the the int and string cases |
59+
| `self.metadata.name == 'singleton'` | Validate that an object's name matches a specific value (making it a singleton) |
60+
| `self.set1.all(e, !(e in self.set2))` | Validate that two listSets are disjoint |
61+
| `self.names.size() == self.details.size() && self.names.all(n, n in self.details)` | Validate the 'details' map is keyed by the items in the 'names' listSet |
62+
63+
## CEL community libraries
64+
65+
Kubernetes CEL expressions have access to the following CEL community libraries:
66+
67+
- CEL standard functions, defined in the [list of standard definitions](https://github.com/google/cel-spec/blob/master/doc/langdef.md#list-of-standard-definitions)
68+
- CEL standard [macros](https://github.com/google/cel-spec/blob/v0.7.0/doc/langdef.md#macros)
69+
- CEL [extended string function library](https://pkg.go.dev/github.com/google/cel-go/ext#Strings)
70+
71+
## Kubernetes CEL libraries
72+
73+
In additional to the CEL community libraries, Kubernetes includes CEL libraries
74+
that are available everywhere CEL is used in Kubernetes.
75+
76+
### Kubernetes list library
77+
78+
The list library includes `indexOf` and `lastIndexOf`, which work similar to the
79+
strings functions of the same names. These functions either the first or last
80+
positional index of the provided element in the list.
81+
82+
The list library also includes `min`, `max` and `sum`. Sum is supported on all
83+
number types as well as the duration type. Min and max are supported on all
84+
comparable types.
85+
86+
`isSorted` is also provided as a convenience function and is supported on all
87+
comparable types.
88+
89+
Examples:
90+
91+
{{< table caption="Examples of CEL expressions using list library functions" >}}
92+
| CEL Expression | Purpose |
93+
|------------------------------------------------------------------------------------|-----------------------------------------------------------|
94+
| `names.isSorted()` | Verify that a list of names is kept in alphabetical order |
95+
| `items.map(x, x.weight).sum() == 1.0` | Verify that the "weights" of a list of objects sum to 1.0 |
96+
| `lowPriorities.map(x, x.priority).max() < highPriorities.map(x, x.priority).min()` | Verify that two sets of priorities do not overlap |
97+
| `names.indexOf('should-be-first') == 1` | Require that the first name in a list if a specific value |
98+
99+
See the [Kubernetes List Library](https://pkg.go.dev/k8s.io/apiextensions-apiserver/pkg/apiserver/schema/cel/library#Lists)
100+
godoc for more information.
101+
102+
### Kubernetes regex library
103+
104+
In addition to the `matches` function provided by the CEL standard library, the
105+
regex library provides `find` and `findAll`, enabling a much wider range of
106+
regex operations.
107+
108+
Examples:
109+
110+
{{< table caption="Examples of CEL expressions using regex library functions" >}}
111+
| CEL Expression | Purpose |
112+
|-------------------------------------------------------------|----------------------------------------------------------|
113+
| `"abc 123".find('[0-9]*')` | Find the first number in a string |
114+
| `"1, 2, 3, 4".findAll('[0-9]*').map(x, int(x)).sum() < 100` | Verify that the numbers in a string sum to less than 100 |
115+
116+
See the [Kubernetes regex library](https://pkg.go.dev/k8s.io/apiextensions-apiserver/pkg/apiserver/schema/cel/library#Regex)
117+
godoc for more information.
118+
119+
### Kubernetes URL library
120+
121+
To make it easier and safer to process URLs, the following functions have been added:
122+
123+
- `isURL(string)` checks if a string is a valid URL according to the [Go's
124+
net/url](https://pkg.go.dev/net/url#URL) package. The string must be an
125+
absolute URL.
126+
- `url(string) URL` converts a string to a URL or results in an error if the
127+
string is not a valid URL.
128+
129+
Once parsed via the `url` function, the resulting URL object has `getScheme`,
130+
`getHost`, `getHostname`, `getPort`, `getEscapedPath` and `getQuery` accessors.
131+
132+
Examples:
133+
134+
{{< table caption="Examples of CEL expressions using URL library functions" >}}
135+
| CEL Expression | Purpose |
136+
|-----------------------------------------------------------------|------------------------------------------------|
137+
| `url('https://example.com:80/').getHost()` | Get the 'example.com:80' host part of the URL. |
138+
| `url('https://example.com/path with spaces/').getEscapedPath()` | Returns '/path%20with%20spaces/' |
139+
140+
See the [Kubernetes URL library](https://pkg.go.dev/k8s.io/apiextensions-apiserver/pkg/apiserver/schema/cel/library#URLs)
141+
godoc for more information.
142+
143+
## Type checking
144+
145+
CEL is a [gradually typed language](https://github.com/google/cel-spec/blob/master/doc/langdef.md#gradual-type-checking).
146+
147+
Some Kubernetes API fields contain fully type checked CEL expressions. For
148+
example, [CustomResourceDefinitions Validation
149+
Rules](/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/#validation-rules)
150+
are fully type checked.
151+
152+
Some Kubernetes API fields contain partially type checked CEL expressions. A
153+
partially type checked expression is an experessions where some of the variables
154+
are statically typed but others are dynamically typed. For example, in the CEL
155+
expressions of
156+
[ValidatingAdmissionPolicies](/docs/reference/access-authn-authz/validating-admission-policy/)
157+
the `request` variable is typed, but the `object` variable is dynamically typed.
158+
As a result, an expression containing `request.namex` would fail type checking
159+
because the `namex` field is not defined. However, `object.namex` would pass
160+
type checking even when the `namex` field is not defined for the resource kinds
161+
that `object` refers to, because `object` is dynamically typed.
162+
163+
The `has()` macro in CEL may be used in CEL expressions to check if a field of a
164+
dynamically typed variable is accessable before attempting to access the field's
165+
value. For example:
166+
167+
```cel
168+
has(object.namex) ? object.namex == 'special' : request.name == 'special'
169+
```
170+
171+
## Type system integration
172+
173+
{{< table caption="Table showing the relationship between OpenAPIv3 types and CEL types" >}}
174+
| OpenAPIv3 type | CEL type |
175+
|----------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------|
176+
| 'object' with Properties | object / "message type" (`type(<object>)` evaluates to `selfType<uniqueNumber>.path.to.object.from.self` |
177+
| 'object' with AdditionalProperties | map |
178+
| 'object' with x-kubernetes-embedded-type | object / "message type", 'apiVersion', 'kind', 'metadata.name' and 'metadata.generateName' are implicitly included in schema |
179+
| 'object' with x-kubernetes-preserve-unknown-fields | object / "message type", unknown fields are NOT accessible in CEL expression |
180+
| x-kubernetes-int-or-string | union of int or string, `self.intOrString < 100 \|\| self.intOrString == '50%'` evaluates to true for both `50` and `"50%"` |
181+
| 'array | list |
182+
| 'array' with x-kubernetes-list-type=map | list with map based Equality & unique key guarantees |
183+
| 'array' with x-kubernetes-list-type=set | list with set based Equality & unique entry guarantees |
184+
| 'boolean' | boolean |
185+
| 'number' (all formats) | double |
186+
| 'integer' (all formats) | int (64) |
187+
| _no equivalent_ | uint (64) |
188+
| 'null' | null_type |
189+
| 'string' | string |
190+
| 'string' with format=byte (base64 encoded) | bytes |
191+
| 'string' with format=date | timestamp (google.protobuf.Timestamp) |
192+
| 'string' with format=datetime | timestamp (google.protobuf.Timestamp) |
193+
| 'string' with format=duration | duration (google.protobuf.Duration) |
194+
195+
Also see: [CEL types](https://github.com/google/cel-spec/blob/v0.6.0/doc/langdef.md#values),
196+
[OpenAPI types](https://swagger.io/specification/#data-types),
197+
[Kubernetes Structural Schemas](/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/#specifying-a-structural-schema).
198+
199+
Equality comparison for arrays with `x-kubernetes-list-type` of `set` or `map` ignores element
200+
order. For example `[1, 2] == [2, 1]` if the arrays represent Kubernetes `set` values.
201+
202+
Concatenation on arrays with `x-kubernetes-list-type` use the semantics of the
203+
list type:
204+
205+
- `set`: `X + Y` performs a union where the array positions of all elements in
206+
`X` are preserved and non-intersecting elements in `Y` are appended, retaining
207+
their partial order.
208+
- `map`: `X + Y` performs a merge where the array positions of all keys in `X`
209+
are preserved but the values are overwritten by values in `Y` when the key
210+
sets of `X` and `Y` intersect. Elements in `Y` with non-intersecting keys are
211+
appended, retaining their partial order.
212+
213+
## Escaping
214+
215+
Only Kubernetes resource property names of the form
216+
`[a-zA-Z_.-/][a-zA-Z0-9_.-/]*` are accessible from CEL. Accessible property
217+
names are escaped according to the following rules when accessed in the
218+
expression:
219+
220+
{{< table caption="Table of CEL identifier escaping rules" >}}
221+
| escape sequence | property name equivalent |
222+
|-------------------|----------------------------------------------------------------------------------------------|
223+
| `__underscores__` | `__` |
224+
| `__dot__` | `.` |
225+
| `__dash__` | `-` |
226+
| `__slash__` | `/` |
227+
| `__{keyword}__` | [CEL **RESERVED** keyword](https://github.com/google/cel-spec/blob/v0.6.0/doc/langdef.md#syntax) |
228+
229+
When you escape any of CEL's **RESERVED** keywords you need to match the exact property name
230+
use the underscore escaping
231+
(for example, `int` in the word `sprint` would not be escaped and nor would it need to be).
232+
233+
Examples on escaping:
234+
235+
{{< table caption="Examples escaped CEL identifiers" >}}
236+
| property name | rule with escaped property name |
237+
|---------------|-----------------------------------|
238+
| `namespace` | `self.__namespace__ > 0` |
239+
| `x-prop` | `self.x__dash__prop > 0` |
240+
| `redact__d` | `self.redact__underscores__d > 0` |
241+
| `string` | `self.startsWith('kube')` |
242+
243+
## Resource constraints
244+
245+
CEL is non-Turing complete and offers a variety of production safety controls to
246+
limit execution time. CEL's _resource constraint_ features provide feedback to
247+
developers about expression complexity and help protect the API server from
248+
excessive resource consumption during evaluation. CEL's resource constraint
249+
features are used to prevent CEL evaluation from consuming excessive API server
250+
resources.
251+
252+
A key element of the resource constraint features is a _cost unit_ that CEL
253+
defines as a way of tracking CPU utilization. Cost units are independant of
254+
system load and hardware. Cost units are also deterministic; for any given CEL
255+
expression and input data, evaluation of the expression by the CEL interpreter
256+
will always result in the same cost.
257+
258+
Many of CEL's core operations have fixed costs. The simplest operations, such as
259+
comparisons (e.g. `<`) have a cost of 1. Some have a higher fixed cost, for
260+
example list literal declarations have a fixed base cost of 40 cost units.
261+
262+
Calls to functions implemented in native code approximate cost based on the time
263+
complexity of the operation. For example: operations that use regular
264+
expressions, such as `match` and `find`, are estimated using an approximated
265+
cost of `length(regexString)*length(inputString)`. The approximated cost
266+
reflects the worst case time complexity of Go's RE2 implementation.
267+
268+
### Runtime cost budget
269+
270+
All CEL expressions evaluated by Kubernetes are constrained by a runtime cost
271+
budget. The runtime cost budget is an estimate of actual CPU utilization
272+
computed by incrementing a cost unit counter while interpreting a CEL
273+
expression. If the CEL interpreter executes too many instructions, the runtime
274+
cost budget will be exceeded, execution of the expressions will be halted, and
275+
an error will result.
276+
277+
Some Kubernetes resources define an additional runtime cost budget that bounds
278+
the execution of multiple expressions. If the sum total of the cost of
279+
expressions exceed the budget, execution of the expressions will be halted, and
280+
an error will result. For example the validation of a custom resource has a
281+
_per-validation_ runtime cost budget for all [Validation
282+
Rules](https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/#validation-rules)
283+
evaluated to validate the custom resource.
284+
285+
### Estimated cost limits
286+
287+
For some Kubernetes resources, the API server may also check if worst case
288+
estimated running time of CEL expressions would be prohibitively expensive to
289+
execute. If so, the API server prevent the CEL expression from being written to
290+
API resources by rejecting create or update operations containing the CEL
291+
expression to the API resources. This feature offers a stronger assurance that
292+
CEL expressions written to the API resource will be evaluate at runtime without
293+
exceeding the runtime cost budget.

0 commit comments

Comments
 (0)