Skip to content

Commit 8f0fdce

Browse files
k8s-ci-robotk8s-ci-robot
authored
Merge pull request kubernetes#1960 from andrewsykim/service-lb-class
KEP-1959: Service Type=LoadBalancer Class Annotation
2 parents 1b35274 + 6221851 commit 8f0fdce

File tree

2 files changed

+249
-0
lines changed

2 files changed

+249
-0
lines changed

‎keps/sig-cloud-provider/1959-service-lb-class-field/README.md

Lines changed: 217 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,217 @@
1+
# KEP-1959: Service Type=LoadBalancer Class Field
2+
3+
<!-- toc -->
4+
- [Release Signoff Checklist](#release-signoff-checklist)
5+
- [Summary](#summary)
6+
- [Motivation](#motivation)
7+
- [Goals](#goals)
8+
- [Non-Goals](#non-goals)
9+
- [Proposal](#proposal)
10+
- [User Stories (Optional)](#user-stories-optional)
11+
- [Story 1](#story-1)
12+
- [Story 2](#story-2)
13+
- [Risks and Mitigations](#risks-and-mitigations)
14+
- [Design Details](#design-details)
15+
- [Test Plan](#test-plan)
16+
- [Graduation Criteria](#graduation-criteria)
17+
- [Upgrade / Downgrade Strategy](#upgrade--downgrade-strategy)
18+
- [Version Skew Strategy](#version-skew-strategy)
19+
- [Implementation History](#implementation-history)
20+
- [Drawbacks](#drawbacks)
21+
- [Alternatives](#alternatives)
22+
- [ServiceClass Resource](#serviceclass-resource)
23+
- [Generic Annotation](#generic-annotation)
24+
- [Provider-Specific Annotations](#provider-specific-annotations)
25+
- [Infrastructure Needed (Optional)](#infrastructure-needed-optional)
26+
<!-- /toc -->
27+
28+
## Release Signoff Checklist
29+
30+
Items marked with (R) are required *prior to targeting to a milestone / release*.
31+
32+
- [ ] (R) Enhancement issue in release milestone, which links to KEP dir in [kubernetes/enhancements] (not the initial KEP PR)
33+
- [ ] (R) KEP approvers have approved the KEP status as `implementable`
34+
- [ ] (R) Design details are appropriately documented
35+
- [ ] (R) Test plan is in place, giving consideration to SIG Architecture and SIG Testing input
36+
- [ ] (R) Graduation criteria is in place
37+
- [ ] (R) Production readiness review completed
38+
- [ ] Production readiness review approved
39+
- [ ] "Implementation History" section is up-to-date for milestone
40+
- [ ] User-facing documentation has been created in [kubernetes/website], for publication to [kubernetes.io]
41+
- [ ] Supporting documentation—e.g., additional design documents, links to mailing list discussions/SIG meetings, relevant PRs/issues, release notes
42+
43+
[kubernetes.io]: https://kubernetes.io/
44+
[kubernetes/enhancements]: https://git.k8s.io/enhancements
45+
[kubernetes/kubernetes]: https://git.k8s.io/kubernetes
46+
[kubernetes/website]: https://git.k8s.io/website
47+
48+
## Summary
49+
50+
When Service Type=LoadBalancer is enabled by a Kubernetes cloud provider, it is a global
51+
configuration that applies for all Service Type=LoadBalancer resources in a given cluster.
52+
This becomes problematic if users want to leverage multiple Service Type=LoadBalancer
53+
implementations in a cluster.
54+
55+
The new [Services APIs](https://github.com/kubernetes-sigs/service-apis) addresses this already
56+
with the GatewayClass resource. However, until Gateway/GatewayClass APIs become mature, we should
57+
support similar functionality for Services of Type=LoadBalancer. Introducing a new resource like
58+
`ServiceClass` is probably not worthwhile given that there are new APIs already in development.
59+
This KEP proposes a light-weight approach for Service Type=LoadBalancer by introducing a Service
60+
field `service.spec.loadBalancerClass`.
61+
62+
## Motivation
63+
64+
The main use-case for this feature is being able to support multiple Service Type=LoadBalancer
65+
implementations in a cluster, as different workloads may want to leverage different loadbalancer
66+
providers based on efficiency, availability, cost and other factors.
67+
68+
For example, a cluster admin may want to use a public load balancer from a cloud provider
69+
for workloads that must be assigned a publically routable address, but they may want to
70+
enable a lower-cost solution for workloads that are only internally accessible.
71+
72+
### Goals
73+
74+
* allow users to opt-out of the Service Type=LoadBalancer implementation by the cloud provider.
75+
* allow multiple implementations of Service Type=LoadBalancer in a cluster.
76+
* prevent every cloud provider from implementing a custom "opt-out" annotation for their load balancer.
77+
78+
### Non-Goals
79+
80+
* performance improvements for Service Type=LoadBalancer.
81+
* changing any other existing behaviors for Service Type=LoadBalancer aside from being able
82+
to disabling it from the cloud provider.
83+
84+
## Proposal
85+
86+
This KEP proposes to add a new field `spec.loadBalancerClass` in Service which allows for
87+
multiple implementations of Service Type=LoadBalancer in a cluster.
88+
89+
### User Stories (Optional)
90+
91+
#### Story 1
92+
93+
As a cluster admin:
94+
* I want to use my cloud provider's public load balancer service for applications that require
95+
public ingress.
96+
* I want to use my own load balancing solution for any applications that only talk internally
97+
within my own network because I want to save costs.
98+
99+
#### Story 2
100+
101+
As an application developer:
102+
* I MUST use a hardware-based loadbalancer for certain applications due to specific protocols
103+
only available there.
104+
* I want to use the cloud provider's default load balancer for any applications that do not
105+
rely on protocols from hardware load balancers.
106+
107+
### Risks and Mitigations
108+
109+
Many cloud providers today support an "opt-out" annotation for this behavior. The annotation is specific
110+
to the cloud provider. Introduction of the `loadBalancerClass` field at this point would mean that
111+
cloud providers need to start accounting for both existing annotations and the new field.
112+
113+
## Design Details
114+
115+
Introduce a new field to Service `spec.loadBalancerClass`.
116+
117+
If the field `spec.loadBalancerClass` is not set, the existing cloud provider will assume
118+
ownership of the Service Type=LoadBalancer resource. This is required to not break existing clusters
119+
that assume Service Type=LoadBalancer is always managed by the cloud provider.
120+
121+
Required API changes:
122+
```go
123+
// ServiceSpec describes the attributes that a user creates on a service.
124+
type ServiceSpec struct {
125+
...
126+
...
127+
128+
// loadBalancerClass is the name of the load balancer implementation this Service belongs to.
129+
// This field can only be set when the Service type is 'LoadBalancer'. If not set, the default load
130+
// balancer implementation is used, today this is typically done through the cloud provider integration,
131+
// but should apply for any default implementation. If set, it is assumed that a load balancer
132+
// implementation is watching for Services with a matching class name. Any default load balancer
133+
// implementation (e.g. cloud providers) should ignore Services that set this field.
134+
// +optional
135+
LoadBalancerClassName string `json:"loadBalancerClass,omitempty"`
136+
}
137+
```
138+
139+
* `loadBalancerClass` will be immutable only when the Service type is `LoadBalancer`, this way existing and future implementations
140+
do not have to worry about handling Services that change the class name. The class name is mutable and can be cleared when the
141+
type changes.
142+
* `loadBalancerClass` will be validated against label-style format.
143+
* the `loadBalancerClass` field will be feature gated. The field will be dropped during API strategy unless
144+
the feature gate is enabled.
145+
146+
Required updates to service controller:
147+
* if the class field is NOT set for a Service, allow the cloud provider to reconcile the load balancer.
148+
* if the class annotation IS set for a Service, skip reconciliation of the Service from the cloud provider.
149+
150+
### Test Plan
151+
152+
Unit tests:
153+
* test that service controller does not call the cloud provider if the class field is set.
154+
* test API strategy to ensure the `loadBalancerClass` field is dropped unless the feature gate is enabled
155+
or an existing Service has the field set.
156+
* test API validation for immutability.
157+
158+
Integration tests:
159+
* test that the class field is propoerly cleared/validated when the Service type changes to and from `LoadBalancer`.
160+
161+
E2E tests:
162+
* test that creating a Service with an unknown class name results in no load balancer being created for a Service.
163+
164+
### Graduation Criteria
165+
166+
Alpha:
167+
* the `loadBalancerClass` field is added to Service with an alpha feature gate.
168+
* when enabled, service controller will ignore Service LBs with a non-empty class name.
169+
* unit tests for service controller.
170+
* unit tests for API strategy (drop disabled fields).
171+
172+
### Upgrade / Downgrade Strategy
173+
174+
* Usage of `loadBalancerClass` will be off by default during the alpha stage but can handle existing Services that
175+
has the field set already. This ensures apiserver can handle the new field on downgrade.
176+
* On upgrade, if the feature gate is enabled, there should be no changes since the default behavior has not changed
177+
(service controller calls the cloud provider to reconcile load balancers).
178+
179+
### Version Skew Strategy
180+
181+
Since this feature will be alpha for at least 1 release, an n-1 kube-controller-manager or cloud-controller-manager should
182+
handle enablement of this feature if a new apiserver enabled it.
183+
184+
## Implementation History
185+
186+
- the `Summary`, `Motivation`, `Proposal` and `Design Details` sections was merged, signaling SIG acceptance
187+
188+
## Drawbacks
189+
190+
* Added complexity to Service.
191+
* In **most** clusters, a single Service Type=LoadBalancer implementation from the cloud provider is sufficient.
192+
193+
## Alternatives
194+
195+
### ServiceClass Resource
196+
197+
Instead of a field specifying the name of the implemmentation, the class name can reference the name of a class resource
198+
similar to GatewayClass and IngressClass. This would enable more expressive configuration per load balancer implementation.
199+
200+
### Generic Annotation
201+
202+
A generic annotation can be used to store the class name. This is avoided since there would be no way to introduce
203+
the annotation in a safe way and we can't enforce immutability for annotations.
204+
205+
### Provider-Specific Annotations
206+
207+
Instead of a generic Kubernetes annotation read by service controller, each cloud provider could implement
208+
their own "skip this Service"-like logic with custom annotations. Given that many cloud providers have been
209+
asking for this feature, a generic field used across all providers may be more beneficial.
210+
211+
## Infrastructure Needed (Optional)
212+
213+
<!--
214+
Use this section if you need things from the project/SIG. Examples include a
215+
new subproject, repos requested, or GitHub details. Listing these here allows a
216+
SIG to get the process for these resources started right away.
217+
-->

‎keps/sig-cloud-provider/1959-service-lb-class-field/kep.yaml

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
title: Service Type=LoadBalancer Class Field
2+
kep-number: 1959
3+
authors:
4+
- "@andrewsykim"
5+
owning-sig: sig-cloud-provider
6+
participating-sigs:
7+
- sig-network
8+
status: implementable
9+
creation-date: 2020-11-02
10+
reviewers:
11+
- "@bowei"
12+
- "@cheftako"
13+
- "@thockin"
14+
approvers:
15+
- "@bowei"
16+
- "@cheftako"
17+
- "@thockin"
18+
prr-approvers:
19+
- "@johnbelamaric"
20+
21+
# The target maturity stage in the current dev cycle for this KEP.
22+
stage: alpha
23+
24+
# The most recent milestone for which work toward delivery of this KEP has been
25+
# done. This can be the current (upcoming) milestone, if it is being actively
26+
# worked on.
27+
latest-milestone: "v1.21"
28+
29+
# The milestone at which this feature was, or is targeted to be, at each stage.
30+
milestone:
31+
alpha: "v1.21"
32+

0 commit comments

Comments
 (0)