Skip to content

Commit 2232222

Browse files
committed
Replace API name ServiceCIDRConfig by ServiceCIDR
1 parent d7560d9 commit 2232222

File tree

1 file changed

+44
-44
lines changed
  • keps/sig-network/1880-multiple-service-cidrs

1 file changed

+44
-44
lines changed

keps/sig-network/1880-multiple-service-cidrs/README.md

Lines changed: 44 additions & 44 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@
1818
- [Current implementation details](#current-implementation-details)
1919
- [New allocation model](#new-allocation-model)
2020
- [The kube-apiserver bootstrap process and the service-cidr flags](#the-kube-apiserver-bootstrap-process-and-the-service-cidr-flags)
21-
- [The special "default" ServiceCIDRConfig](#the-special-default-servicecidrconfig)
21+
- [The special "default" ServiceCIDR](#the-special-default-servicecidr)
2222
- [Service IP Allocation](#service-ip-allocation)
2323
- [Service IP Reservation](#service-ip-reservation)
2424
- [Edge cases](#edge-cases)
@@ -138,27 +138,27 @@ Implement a new allocation logic for Services IPs that:
138138

139139
## Proposal
140140

141-
The proposal is to implement a new allocator logic that uses 2 new API Objects: ServiceCIDRConfig
141+
The proposal is to implement a new allocator logic that uses 2 new API Objects: ServiceCIDR
142142
and IPAddress, and allow users to dynamically increase the number of Services IPs available by
143-
creating new ServiceCIDRConfigs.
143+
creating new ServiceCIDRs.
144144

145-
The new allocator will be able to "automagically" consume IPs from any ServiceCIDRConfig available, we can
145+
The new allocator will be able to "automagically" consume IPs from any ServiceCIDR available, we can
146146
think about this model, as the same as adding more disks to a Storage system to increase the
147147
capacity.
148148

149149
To simplify the model, make it backwards compatible and to avoid that it can evolve into something
150150
different and collide with other APIs, like Gateway APIs, we are adding the following constraints:
151151

152-
- a ServiceCIDRConfig will be immutable after creation (to be revisited before Beta).
153-
- a ServiceCIDRConfig can only be deleted if there are no Service IP associated to it (enforced by finalizer).
154-
- there can be overlapping ServiceCIDRConfigs.
155-
- the apiserver will periodically ensure that a "default" ServiceCIDRConfig exists to cover the service CIDR flags
152+
- a ServiceCIDR will be immutable after creation (to be revisited before Beta).
153+
- a ServiceCIDR can only be deleted if there are no Service IP associated to it (enforced by finalizer).
154+
- there can be overlapping ServiceCIDRs.
155+
- the apiserver will periodically ensure that a "default" ServiceCIDR exists to cover the service CIDR flags
156156
and the "kubernetes.default" Service.
157-
- any IPAddress existing in a cluster has to belong to a Service CIDR defined by a ServiceCIDRConfig.
157+
- any IPAddress existing in a cluster has to belong to a Service CIDR defined by a ServiceCIDR.
158158
- any Service with a ClusterIP assigned is expected to have always a IPAddress object associated.
159-
- a ServiceCIDRConfig which is being deleted can not be used to allocate new IPs
159+
- a ServiceCIDR which is being deleted can not be used to allocate new IPs
160160

161-
This creates a 1-to-1 relation between Service and IPAddress, and a 1-to-N relation between the ServiceCIDRs defined by the ServiceCIDRConfig and IPAddress. It is important to clarify that overlapping ServiceCIDRConfig are merged in memory, an IPAddress doesn't come from a specific ServiceCIDRConfig object, but "any ServiceCIDRConfig that includes that IP"
161+
This creates a 1-to-1 relation between Service and IPAddress, and a 1-to-N relation between the ServiceCIDRs defined by the ServiceCIDR and IPAddress. It is important to clarify that overlapping ServiceCIDR are merged in memory, an IPAddress doesn't come from a specific ServiceCIDR object, but "any ServiceCIDR that includes that IP"
162162

163163
The new allocator logic can be used by other APIs, like Gateway API.
164164

@@ -252,13 +252,13 @@ multiple apiservers.
252252

253253
The new allocation mode requires:
254254

255-
- 2 new API objects ServiceCIDRConfig and IPAddress in networking.k8s.io/v1alpha1, see <https://groups.google.com/g/kubernetes-sig-api-machinery/c/S0KuN_PJYXY/m/573BLOo4EAAJ>. Both will be protected with finalizers.
255+
- 2 new API objects ServiceCIDR and IPAddress in networking.k8s.io/v1alpha1, see <https://groups.google.com/g/kubernetes-sig-api-machinery/c/S0KuN_PJYXY/m/573BLOo4EAAJ>. Both will be protected with finalizers.
256256
- 1 new allocator implementing current `allocator.Interface`, that runs in each apiserver, and uses the new objects to allocate IPs for Services.
257-
- 1 new controller that participates on the ServiceCIDRConfig deletion, the guarantee that each IPAddresses has
258-
a ServiceCIDRConfig associated. It also handles the special case for the `default` ServiceCIDRConfig.
257+
- 1 new controller that participates on the ServiceCIDR deletion, the guarantee that each IPAddresses has
258+
a ServiceCIDR associated. It also handles the special case for the `default` ServiceCIDR.
259259
- 1 new controller that participates in the IPAddress deletion, that guarantees that each Service IP
260260
allocated has its corresponding IPAddress object, recreating it if necessary.
261-
- 1 new controller that handles the bootstrap process and the default ServiceCIDRConfig.
261+
- 1 new controller that handles the bootstrap process and the default ServiceCIDR.
262262

263263
[![](https://mermaid.ink/img/pako:eNp9UstqwzAQ_BWxx5IcCqUHUwrFacGXYJLeqh620tpV0SNIcsAk_vfKjpNiJ3QvYmdGs7NIBxBOEmQgNIawUlh7NNyyVAPCtuT3StDhhPWV6yZE8kXJQvTK1jeYwD4-52RRvqFRWlFPjk17Rbel00q07G7av7c7Omm7G-HyYrXJna1UfYm5RkOzfEW5f7iGHifQxL0oX6T0FMJ_rhuqmPv6ScfE4VynbszJONxzYE_H5fL4PDaXIVkCUGsnMFLgMLn4t-DcYj23EM5GVHZwgAUY8gaVTA88LMEhfpMhDr1UUoWNjr2yS9JmJ9PoV6mi85BVqAMtAJvotq0VkEXf0Fk0_pNR1f0CyVS2dw)](https://mermaid.live/edit#pako:eNp9UstqwzAQ_BWxx5IcCqUHUwrFacGXYJLeqh620tpV0SNIcsAk_vfKjpNiJ3QvYmdGs7NIBxBOEmQgNIawUlh7NNyyVAPCtuT3StDhhPWV6yZE8kXJQvTK1jeYwD4-52RRvqFRWlFPjk17Rbel00q07G7av7c7Omm7G-HyYrXJna1UfYm5RkOzfEW5f7iGHifQxL0oX6T0FMJ_rhuqmPv6ScfE4VynbszJONxzYE_H5fL4PDaXIVkCUGsnMFLgMLn4t-DcYj23EM5GVHZwgAUY8gaVTA88LMEhfpMhDr1UUoWNjr2yS9JmJ9PoV6mi85BVqAMtAJvotq0VkEXf0Fk0_pNR1f0CyVS2dw)
264264

@@ -275,10 +275,10 @@ and Services.
275275

276276
In order to be completely backwards compatible, the bootstrap process will remain the same, the
277277
difference is that instead of creating a bitmap based on the flags, it will create a new
278-
ServiceCIDRConfig object from the flags (flags configuration removal is out of scope of this KEP)
278+
ServiceCIDR object from the flags (flags configuration removal is out of scope of this KEP)
279279
with a special label `networking.kubernetes.io/service-cidr-from-flags` set to `"true"`.
280280

281-
It now has to handle the possibility of multiple ServiceCIDRConfig with the special label, and
281+
It now has to handle the possibility of multiple ServiceCIDR with the special label, and
282282
also updating the configuration, per example, from single-stack to dual-stack.
283283

284284
The new bootstrap process will be:
@@ -313,11 +313,11 @@ controller on_event:
313313
create it
314314
```
315315

316-
#### The special "default" ServiceCIDRConfig
316+
#### The special "default" ServiceCIDR
317317

318318
The `kubernetes.default` Service is expected to be covered by a valid range. Each apiserver will
319-
ensure that a ServiceCIDRConfig object exists to cover its own flag-defined ranges, so this should
320-
be true in normal cases. If someone were to force-delete the ServiceCIDRConfig covering `kubernetes.default` it
319+
ensure that a ServiceCIDR object exists to cover its own flag-defined ranges, so this should
320+
be true in normal cases. If someone were to force-delete the ServiceCIDR covering `kubernetes.default` it
321321
would be treated the same as any Service in the repair loop, which will generate warnings about
322322
orphaned Service IPs.
323323

@@ -358,48 +358,48 @@ there are different possibilities, that should be resolved before Beta.
358358

359359
```
360360
<<[UNRESOLVED keps/sig-network/3070-reserved-service-ip-range]>>
361-
Option 1: Maintain the same formula and behavior per ServiceCIDRConfig
361+
Option 1: Maintain the same formula and behavior per ServiceCIDR
362362
Option 2: Define a new "allocationMode: (Dynamic | Static | Mixed)" field
363363
Option 3: Define a new "allocationStaticThreshold: int" field
364364
<<[/UNRESOLVED]>>
365365
```
366366

367367
#### Edge cases
368368

369-
Since we have to maintain 3 relationships Services, ServiceCIDRConfigs and IPAddresses, we should be able
369+
Since we have to maintain 3 relationships Services, ServiceCIDRs and IPAddresses, we should be able
370370
to handle edge cases and race conditions.
371371

372-
- Valid Service and IPAddress without ServiceCIDRConfig:
372+
- Valid Service and IPAddress without ServiceCIDR:
373373

374-
This situation can happen if a user forcefully delete the ServiceCIDRConfig, it can be recreated for the
375-
"default" ServiceCIDRConfigs because the information is in the apiserver flags, but for other ServiceCIDRConfigs
374+
This situation can happen if a user forcefully delete the ServiceCIDR, it can be recreated for the
375+
"default" ServiceCIDRs because the information is in the apiserver flags, but for other ServiceCIDRs
376376
that information is no longer available.
377377

378-
Another possible situation is when one ServiceCIDRConfig has been deleted, but the information takes too long
378+
Another possible situation is when one ServiceCIDR has been deleted, but the information takes too long
379379
to reach one of the apiservers, its allocator will still consider the range valid and may allocate one IP
380380
from there. To mitigate this problem, we'll set a grace period of 60 seconds on the servicecidrconfig controller
381381
to remove the finalizer, if an IP address is created during this time we'll be able to block the deletion
382382
and inform the user.
383383

384384
[![](https://mermaid.ink/img/pako:eNp1kjFvwyAQhf8KYm3ixlZVtQyRbNwhW9WsLCe4JEg2uBi3qqL890Jw67hNGBDifffuieNIpVVIGe3xfUAjsdawd9AKQ8IC6a0jFYGe1NigR7JF96El8k39xq3Z6X0CO3BeS92B8YRHHDrdBxQdKf8T9ZyoLpuVUeMOYWomTAIqslyv7zi7mYXkz0WWPz5lq2x1XzykqrSXsZbU7I_1qBobrmzMEghoGisjM7nlReLqs0vJfsuTm0oqj-qyYleCqXPiiRvDzPPOqSnTRb_NK_nU_mAHf3USwtAFbdG1oFWY6TE6CeoP2KKgLBwV7mBovKDCnAI6dCrEf1E6vDxlO2h6XFAYvN1-GUmZdwP-QOO_GKnTN31Vsn4)](https://mermaid.live/edit#pako:eNp1kjFvwyAQhf8KYm3ixlZVtQyRbNwhW9WsLCe4JEg2uBi3qqL890Jw67hNGBDifffuieNIpVVIGe3xfUAjsdawd9AKQ8IC6a0jFYGe1NigR7JF96El8k39xq3Z6X0CO3BeS92B8YRHHDrdBxQdKf8T9ZyoLpuVUeMOYWomTAIqslyv7zi7mYXkz0WWPz5lq2x1XzykqrSXsZbU7I_1qBobrmzMEghoGisjM7nlReLqs0vJfsuTm0oqj-qyYleCqXPiiRvDzPPOqSnTRb_NK_nU_mAHf3USwtAFbdG1oFWY6TE6CeoP2KKgLBwV7mBovKDCnAI6dCrEf1E6vDxlO2h6XFAYvN1-GUmZdwP-QOO_GKnTN31Vsn4)
385385

386-
For any Service and IPAddress that doesn't belong to a ServiceCIDRConfig the controller will raise an event
386+
For any Service and IPAddress that doesn't belong to a ServiceCIDR the controller will raise an event
387387
informing the user, keeping the previous behavior
388388

389389
```go
390390
// cluster IP is out of range
391391
c.recorder.Eventf(&svc, nil, v1.EventTypeWarning, "ClusterIPOutOfRange", "ClusterIPAllocation", "Cluster IP [%v]:%s is not within the service CIDR %s; please recreate service",
392392
```
393393

394-
- Valid Service and ServiceCIDRConfig but not IPAddress
394+
- Valid Service and ServiceCIDR but not IPAddress
395395

396-
It can happen that an user forcefully delete an IPAddress, in this case, the controller will regenerate the IPAddress, as long as a valid ServiceCIDRConfig covers it.
396+
It can happen that an user forcefully delete an IPAddress, in this case, the controller will regenerate the IPAddress, as long as a valid ServiceCIDR covers it.
397397

398398
During this time, there is a chance that an apiserver tries to allocate this IPAddress, with a possible situation where
399399
2 Services has the same IPAddress. In order to avoid it, the Allocator will not delete an IP from its local cache
400400
until it verifies that the consumer associated to that IP has been deleted too.
401401

402-
- Valid IPAddress and ServiceCIDRConfig but no Service
402+
- Valid IPAddress and ServiceCIDR but no Service
403403

404404
The IPAddress will be deleted and an event generated if the controller determines that the IPAddress
405405
is orphan [(see Allocator section)](#Allocator)
@@ -444,24 +444,24 @@ correct, and will start sending events to warn the user that it has to fix/recre
444444
#### API
445445

446446
```go
447-
// ServiceCIDRConfig defines a range of IPs using CIDR format (192.168.0.0/24 or 2001:db2::0/64).
448-
type ServiceCIDRConfig struct {
447+
// ServiceCIDR defines a range of IPs using CIDR format (192.168.0.0/24 or 2001:db2::0/64).
448+
type ServiceCIDR struct {
449449
metav1.TypeMeta `json:",inline"`
450450
metav1.ObjectMeta `json:"metadata,omitempty"`
451451

452-
Spec ServiceCIDRConfigSpec `json:"spec,omitempty"`
452+
Spec ServiceCIDRSpec `json:"spec,omitempty"`
453453
}
454454

455455

456-
// ServiceCIDRConfigSpec describe how the ServiceCIDRConfig's specification looks like.
457-
type ServiceCIDRConfigSpec struct {
456+
// ServiceCIDRSpec describe how the ServiceCIDR's specification looks like.
457+
type ServiceCIDRSpec struct {
458458
// IPv4 is an IPv4 block in CIDR notation "10.0.0.0/8"
459459
IPv4 string `json:"ipv4"`
460460
// IPv6 is an IPv6 block in CIDR notation "fd12:3456:789a:1::/64"
461461
IPv6 string `json:"ipv6"`
462462
}
463463

464-
// IPAddress represents an IP used by Kubernetes associated to an ServiceCIDRConfig.
464+
// IPAddress represents an IP used by Kubernetes associated to an ServiceCIDR.
465465
// The name of the object is the IP address in canonical format.
466466
type IPAddress struct {
467467
metav1.TypeMeta `json:",inline"`
@@ -514,10 +514,10 @@ type Interface interface {
514514
}
515515
```
516516

517-
This allocator will have an informer watching Services, ServiceCIDRConfigs and IPAddresses, so it can have locally the
517+
This allocator will have an informer watching Services, ServiceCIDRs and IPAddresses, so it can have locally the
518518
information needed to assign new IP addresses to Services.
519519

520-
IPAddresses can only be allocated from ServiceCIDRConfigs that are available and not being deleted.
520+
IPAddresses can only be allocated from ServiceCIDRs that are available and not being deleted.
521521

522522
The uniqueness of an IPAddress is guaranteed by the apiserver, since trying to create an IP
523523
address that already exist will fail.
@@ -533,7 +533,7 @@ This is a very core and critical change, it has to be thoroughly tested on diffe
533533

534534
API objects must have unit tests for defaulting and validation and integration tests exercising the
535535
different operations and fields permutations, with both positive and negative cases: Special
536-
attention to cross-reference validation problems, like create IPAddress reference wrong ServiceCIDRConfig or
536+
attention to cross-reference validation problems, like create IPAddress reference wrong ServiceCIDR or
537537
invalid or missing
538538

539539
Controllers must have unit tests and integration tests covering all possible race conditions.
@@ -616,9 +616,9 @@ We expect no non-infra related flakes in the last month as a GA graduation crite
616616
- API stability, no changes on new types and behaviors:
617617
- ServiceCIDR immutability
618618
- default IPFamily
619-
- two or one IP families per ServiceCIDRConfig
619+
- two or one IP families per ServiceCIDR
620620
- Gather feedback from developers and users
621-
- Document and add more complex testing scenarios: scaling out ServiceCIDRConfigs, ...
621+
- Document and add more complex testing scenarios: scaling out ServiceCIDRs, ...
622622
- Additional tests are in Testgrid and linked in KEP
623623
- Scale test to O(10K) services and O(1K) ranges
624624
- Improve performance on the allocation logic, O(1) for allocating a free address
@@ -664,7 +664,7 @@ Example:
664664

665665
- flags set to 10.0.0.0/20
666666
- upgrade to 1.25 with alpha gate
667-
- apiserver create a default ServiceCIDRConfig object for 10.0.0.0/20
667+
- apiserver create a default ServiceCIDR object for 10.0.0.0/20
668668
- user creates a new ServiceCIDR for 192.168.1.0/24
669669
- create a Service which gets 192.168.1.1
670670
- rollback or disable the gate
@@ -678,7 +678,7 @@ Example:
678678
###### How can this feature be enabled / disabled in a live cluster?
679679

680680
- [x] Feature gate (also fill in values in `kep.yaml`)
681-
- Feature gate name: ServiceCIDRConfig
681+
- Feature gate name: ServiceCIDR
682682
- Components depending on the feature gate: kube-apiserver, kube-controller-manager
683683

684684
###### Does enabling the feature change any default behavior?
@@ -891,7 +891,7 @@ Think about adding additional work or introducing new steps in between
891891

892892
###### Will enabling / using this feature result in non-negligible increase of resource usage (CPU, RAM, disk, IO, ...) in any components?
893893

894-
The apiservers will increase the memory usage because they have to keep a local informer with the new objects ServiceCIDRConfig and IPAddress.
894+
The apiservers will increase the memory usage because they have to keep a local informer with the new objects ServiceCIDR and IPAddress.
895895

896896
### Troubleshooting
897897

0 commit comments

Comments
 (0)