Skip to content

Commit e1b2075

Browse files
LiorLiebermanLiorLieberman
authored
KEP-3726: standard-application-protocols (#3727)
* 3726-standard-application-protocols KEP * fix * fix * add proposed api change + criterias to add new protocols to the list * update toc * fix typo * remove undesired change * more changes * address feedback * update toc * add unresolved section and mark checklist steps * fix * undo fix
1 parent 0e2da7a commit e1b2075

File tree

2 files changed

+328
-0
lines changed

2 files changed

+328
-0
lines changed

‎keps/sig-network/3726-standard-application-protocols/README.md

Lines changed: 294 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,294 @@
1+
# KEP-3726: Standard Application Protocols
2+
3+
<!--
4+
This is the title of your KEP. Keep it short, simple, and descriptive. A good
5+
title can help communicate what the KEP is and should be considered as part of
6+
any review.
7+
-->
8+
9+
<!--
10+
A table of contents is helpful for quickly jumping to sections of a KEP and for
11+
highlighting any additional information provided beyond the standard KEP
12+
template.
13+
14+
Ensure the TOC is wrapped with
15+
<code>&lt;!-- toc --&rt;&lt;!-- /toc --&rt;</code>
16+
tags, and then generate with `hack/update-toc.sh`.
17+
-->
18+
19+
<!-- toc -->
20+
- [Release Signoff Checklist](#release-signoff-checklist)
21+
- [Summary](#summary)
22+
- [Motivation](#motivation)
23+
- [Goals](#goals)
24+
- [Non-Goals](#non-goals)
25+
- [Proposal](#proposal)
26+
- [New Standard Protocols](#new-standard-protocols)
27+
- [Risks and Mitigations](#risks-and-mitigations)
28+
- [Design Details](#design-details)
29+
- [Adding new protocols](#adding-new-protocols)
30+
- [Followup work](#followup-work)
31+
- [Documentation change](#documentation-change)
32+
- [Test Plan](#test-plan)
33+
- [Graduation Criteria](#graduation-criteria)
34+
- [Upgrade / Downgrade Strategy](#upgrade--downgrade-strategy)
35+
- [Version Skew Strategy](#version-skew-strategy)
36+
- [Production Readiness Review Questionnaire](#production-readiness-review-questionnaire)
37+
- [Feature Enablement and Rollback](#feature-enablement-and-rollback)
38+
- [Rollout, Upgrade and Rollback Planning](#rollout-upgrade-and-rollback-planning)
39+
- [Monitoring Requirements](#monitoring-requirements)
40+
- [Dependencies](#dependencies)
41+
- [Scalability](#scalability)
42+
- [Troubleshooting](#troubleshooting)
43+
- [Implementation History](#implementation-history)
44+
- [Drawbacks](#drawbacks)
45+
- [Alternatives](#alternatives)
46+
<!-- /toc -->
47+
48+
## Release Signoff Checklist
49+
50+
<!--
51+
**ACTION REQUIRED:** In order to merge code into a release, there must be an
52+
issue in [kubernetes/enhancements] referencing this KEP and targeting a release
53+
milestone **before the [Enhancement Freeze](https://git.k8s.io/sig-release/releases)
54+
of the targeted release**.
55+
56+
For enhancements that make changes to code or processes/procedures in core
57+
Kubernetes—i.e., [kubernetes/kubernetes], we require the following Release
58+
Signoff checklist to be completed.
59+
60+
Check these off as they are completed for the Release Team to track. These
61+
checklist items _must_ be updated for the enhancement to be released.
62+
-->
63+
64+
Items marked with (R) are required *prior to targeting to a milestone / release*.
65+
66+
- [x] (R) Enhancement issue in release milestone, which links to KEP dir in [kubernetes/enhancements] (not the initial KEP PR)
67+
- [x] (R) KEP approvers have approved the KEP status as `implementable`
68+
- [x] (R) Design details are appropriately documented
69+
- [x] (R) Test plan is in place, giving consideration to SIG Architecture and SIG Testing input (including test refactors)
70+
- [x] e2e Tests for all Beta API Operations (endpoints)
71+
- [x] (R) Ensure GA e2e tests meet requirements for [Conformance Tests](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/conformance-tests.md)
72+
- [x] (R) Minimum Two Week Window for GA e2e tests to prove flake free
73+
- [x] (R) Graduation criteria is in place
74+
- [x] (R) [all GA Endpoints](https://github.com/kubernetes/community/pull/1806) must be hit by [Conformance Tests](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/conformance-tests.md)
75+
- [x] (R) Production readiness review completed
76+
- [x] (R) Production readiness review approved
77+
- [x] "Implementation History" section is up-to-date for milestone
78+
- [ ] User-facing documentation has been created in [kubernetes/website], for publication to [kubernetes.io]
79+
- [ ] Supporting documentation—e.g., additional design documents, links to mailing list discussions/SIG meetings, relevant PRs/issues, release notes
80+
81+
<!--
82+
**Note:** This checklist is iterative and should be reviewed and updated every time this enhancement is being considered for a milestone.
83+
-->
84+
85+
[kubernetes.io]: https://kubernetes.io/
86+
[kubernetes/enhancements]: https://git.k8s.io/enhancements
87+
[kubernetes/kubernetes]: https://git.k8s.io/kubernetes
88+
[kubernetes/website]: https://git.k8s.io/website
89+
90+
## Summary
91+
92+
There are cases where implementations implement different things for the same application protocol names. That has already caused issues for certain implementations to interoperate, with the possibility of more in the future. (See the example with GKE and Istio under [Motivation](#motivation))
93+
94+
This KEP suggests a different description for the `AppProtocol` field and proposes to declare standard Kubernetes protocol names for common protocols that are not IANA service names.
95+
96+
97+
## Motivation
98+
99+
The lack of direct support for specifying application protocols for ports and the widespread use of implementation-specific annotations to mitigate it had led Kubernetes to add the `AppProtocol` field to the port spec.
100+
101+
While this is a good solution - we never came with recommended standards other than [IANA standard service names](https://www.iana.org/assignments/service-names) or a domain prefixed protocol.
102+
103+
This loose definition has led us to;
104+
1. Have instances where implementations do different things for common protocols.
105+
2. Have no support for implementations interoperability with different domain prefixed protocols (or a mix domain prefixed and non prefixed protocol) for the same port.
106+
107+
One good example for the first bullet is `HTTP2`.
108+
* In GKE you can use `appProtocol: HTTP2` and it will describe HTTP2 over TLS (https://cloud.google.com/kubernetes-engine/docs/how-to/secure-gateway#load-balancer-tls).
109+
* While in Istio it will be h2c (HTTP2 over cleartext).
110+
111+
That creates a problem where users with GKE and Istio in their cluster can have very different behaviors for the same `appProtocol` value.
112+
113+
114+
### Goals
115+
116+
* Rephrase AppProtocol field description
117+
* Build consensus around how common (non IANA service names) should be implemented
118+
* Help the broader community and specifically implementations to interoperate better
119+
* Provide short and clear documentation for how AppProtocol values should be implemented
120+
* Update the appProtocol user documentation respectively
121+
122+
123+
### Non-Goals
124+
125+
* Validate appProtocol values
126+
* Monitor appProtocol implementations
127+
* Support multiple AppProtocols values for the same port to improve interoperability (suggested as a followup work though)
128+
129+
130+
## Proposal
131+
132+
~~Kubernetes supports the `appProtocol` field to provide a way to specify an application protocol for each Service port.~~
133+
134+
Kubernetes `appProtocol` field is used as a hint for implementations to configure the protocol used between the implementation and the application it exposes.
135+
136+
The [documentation](https://kubernetes.io/docs/concepts/services-networking/service/#application-protocol) for this field says that:
137+
138+
```Values should either be IANA standard service names or domain prefixed names such as mycompany.com/my-custom-protocol.```
139+
140+
This KEP proposes to declare standard Kubernetes protocol names for common protocols that are not IANA standard service names.
141+
142+
Those common protocols will be well defined strings prefixed with ‘k8s.io’.
143+
`k8s.io/h2c` as an example.
144+
145+
### New Standard Protocols
146+
- 'k8s.io/http2'
147+
- 'k8s.io/grpc'
148+
- 'k8s.io/tcp'
149+
150+
### Risks and Mitigations
151+
152+
There are no real “risks”, primary concerns are:
153+
1. End users will not migrate to `k8s.io/<>` values
154+
2. It will take long time to implementations align with the new standards
155+
3. We have no easy way to monitor who is aligned and who is not
156+
157+
158+
## Design Details
159+
160+
At first, the collection of standard protocols is going to live in `ServicePort` and `EndpointPort` as part of the AppProtocol description.
161+
162+
We might revisit this decision in the future and suggest an alternative location based on the number of standard protocols we support.
163+
164+
Proposed changes to the API spec:
165+
166+
```go
167+
type ServicePort struct {
168+
...
169+
...
170+
171+
// Used as a hint for implementations to
172+
// configure the protocol used between the
173+
// implementation and the application it exposes.
174+
// This field follows standard Kubernetes label syntax.
175+
// Valid values are either:
176+
//
177+
// * Un-prefixed protocol names - reserved for IANA standard service names (as per
178+
// RFC-6335 and https://www.iana.org/assignments/service-names).
179+
//
180+
// * Kubernetes standard names:
181+
// * 'k8s.io/http2' - http2 over cleartext, aka 'h2c'. https://www.rfc-editor.org/rfc/rfc7540
182+
// * 'k8s.io/grpc' - grpc traffic - see https://github.com/grpc/grpc/blob/v1.51.1/doc/PROTOCOL-HTTP2.md
183+
// * 'k8s.io/tcp' - plain tcp traffic
184+
//
185+
// * Other protocols should use prefixed names such as
186+
// mycompany.com/my-custom-protocol.
187+
// +optional
188+
AppProtocol *string
189+
```
190+
191+
same wording for type `EndpointPort`
192+
193+
### Adding new protocols
194+
195+
In order to be included in the collection, a new protocol must:
196+
* Not be an [IANA standard service name](https://www.iana.org/assignments/service-names)
197+
* Run on top of L4 protocol supported by Kubernetes Service
198+
* Be supported in two or more implementations
199+
* Be well defined and broadly used
200+
201+
<<[UNRESOLVED sig-network ]>>
202+
203+
if we require two implementations to support a protocol before we include it in the standard collection (i.e k8s.io prefixed collection) we create a situation where we force them to support their own domain prefixed values and k8s.io-prefixed values, have their users migrate to the k8s.io values once they are included, and also the k8s.io ones might end up not be quite the same definition as the implementation specific ones (as we see in the GKE & Istio example).
204+
205+
The proposed followup work might address this problem also when we turn the field into a list
206+
207+
<<[/UNRESOLVED]>>
208+
209+
210+
### Followup work
211+
To support implementations interoperability with different domain prefixed protocols (or a mix domain prefixed and non prefixed protocol) for the same port we need to turn `AppProtocol` to a list.
212+
213+
It is likely to be an API change but design details TBD.
214+
215+
### Documentation change
216+
217+
[kubernetes website](https://github.com/kubernetes/website/blob/main/content/en/docs/concepts/services-networking/service.md#application-protocol) will be changed accordingly
218+
219+
### Test Plan
220+
221+
N/A
222+
223+
This KEP does not plan to change code, just documentation.
224+
225+
### Graduation Criteria
226+
227+
### Upgrade / Downgrade Strategy
228+
229+
N/A
230+
231+
This KEP does not plan to change code, just documentation.
232+
233+
### Version Skew Strategy
234+
235+
N/A
236+
237+
This KEP does not plan to change code, just documentation.
238+
239+
## Production Readiness Review Questionnaire
240+
241+
242+
### Feature Enablement and Rollback
243+
244+
N/A
245+
246+
This KEP does not plan to change code, just documentation.
247+
248+
249+
### Rollout, Upgrade and Rollback Planning
250+
251+
N/A
252+
253+
This KEP does not plan to change code, just documentation.
254+
255+
### Monitoring Requirements
256+
257+
N/A
258+
259+
This KEP does not plan to change code, just documentation.
260+
261+
### Dependencies
262+
263+
N/A
264+
265+
This KEP does not plan to change code, just documentation.
266+
267+
### Scalability
268+
269+
N/A
270+
271+
This KEP does not plan to change code, just documentation.
272+
273+
### Troubleshooting
274+
275+
N/A
276+
277+
This KEP does not plan to change code, just documentation.
278+
279+
## Implementation History
280+
281+
N/A
282+
283+
## Drawbacks
284+
285+
* The collection of the standard protocols can become stale fairly quick when new protocols are implemented before we decide to declare them as part of k8s.io common collection. That can lead to a the current state again where implementations already implement support without a prefix (although they should not) OR with a domain prefix.
286+
287+
288+
## Alternatives
289+
290+
<!--
291+
What other approaches did you consider, and why did you rule them out? These do
292+
not need to be as detailed as the proposal, but should include enough
293+
information to express the idea and why it was not acceptable.
294+
-->

‎keps/sig-network/3726-standard-application-protocols/kep.yaml

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
title: KEP Template
2+
kep-number: 3726
3+
authors:
4+
- "@LiorLieberman"
5+
owning-sig: sig-network
6+
7+
status: provisional
8+
creation-date: 2023-01-11
9+
reviewers:
10+
- "@aojea"
11+
- "@robscott"
12+
- "@howardjohn"
13+
- "@danwinship"
14+
approvers:
15+
- "@thockin"
16+
17+
# The target maturity stage in the current dev cycle for this KEP.
18+
stage: stable
19+
20+
# The most recent milestone for which work toward delivery of this KEP has been
21+
# done. This can be the current (upcoming) milestone, if it is being actively
22+
# worked on.
23+
latest-milestone: "v1.27"
24+
25+
# The milestone at which this feature was, or is targeted to be, at each stage.
26+
milestone:
27+
stable: "v1.27"
28+
29+
see-also: []
30+
replaces: []
31+
32+
feature-gates: []
33+
disable-supported: false
34+
metrics: []

0 commit comments

Comments
 (0)