You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
-[The kube-apiserver bootstrap process and the service-cidr flags](#the-kube-apiserver-bootstrap-process-and-the-service-cidr-flags)
21
-
-[The special "default"ServiceCIDRConfig](#the-special-default-servicecidrconfig)
21
+
-[The special "default"ServiceCIDR](#the-special-default-servicecidr)
22
22
-[Service IP Allocation](#service-ip-allocation)
23
23
-[Service IP Reservation](#service-ip-reservation)
24
24
-[Edge cases](#edge-cases)
@@ -138,27 +138,27 @@ Implement a new allocation logic for Services IPs that:
138
138
139
139
## Proposal
140
140
141
-
The proposal is to implement a new allocator logic that uses 2 new API Objects: ServiceCIDRConfig
141
+
The proposal is to implement a new allocator logic that uses 2 new API Objects: ServiceCIDR
142
142
and IPAddress, and allow users to dynamically increase the number of Services IPs available by
143
-
creating new ServiceCIDRConfigs.
143
+
creating new ServiceCIDRs.
144
144
145
-
The new allocator will be able to "automagically" consume IPs from any ServiceCIDRConfig available, we can
145
+
The new allocator will be able to "automagically" consume IPs from any ServiceCIDR available, we can
146
146
think about this model, as the same as adding more disks to a Storage system to increase the
147
147
capacity.
148
148
149
149
To simplify the model, make it backwards compatible and to avoid that it can evolve into something
150
150
different and collide with other APIs, like Gateway APIs, we are adding the following constraints:
151
151
152
-
- a ServiceCIDRConfig will be immutable after creation (to be revisited before Beta).
153
-
- a ServiceCIDRConfig can only be deleted if there are no Service IP associated to it (enforced by finalizer).
154
-
- there can be overlapping ServiceCIDRConfigs.
155
-
- the apiserver will periodically ensure that a "default" ServiceCIDRConfig exists to cover the service CIDR flags
152
+
- a ServiceCIDR will be immutable after creation (to be revisited before Beta).
153
+
- a ServiceCIDR can only be deleted if there are no Service IP associated to it (enforced by finalizer).
154
+
- there can be overlapping ServiceCIDRs.
155
+
- the apiserver will periodically ensure that a "default" ServiceCIDR exists to cover the service CIDR flags
156
156
and the "kubernetes.default" Service.
157
-
- any IPAddress existing in a cluster has to belong to a Service CIDR defined by a ServiceCIDRConfig.
157
+
- any IPAddress existing in a cluster has to belong to a Service CIDR defined by a ServiceCIDR.
158
158
- any Service with a ClusterIP assigned is expected to have always a IPAddress object associated.
159
-
- a ServiceCIDRConfig which is being deleted can not be used to allocate new IPs
159
+
- a ServiceCIDR which is being deleted can not be used to allocate new IPs
160
160
161
-
This creates a 1-to-1 relation between Service and IPAddress, and a 1-to-N relation between the ServiceCIDRs defined by the ServiceCIDRConfig and IPAddress. It is important to clarify that overlapping ServiceCIDRConfig are merged in memory, an IPAddress doesn't come from a specific ServiceCIDRConfig object, but "any ServiceCIDRConfig that includes that IP"
161
+
This creates a 1-to-1 relation between Service and IPAddress, and a 1-to-N relation between the ServiceCIDRs defined by the ServiceCIDR and IPAddress. It is important to clarify that overlapping ServiceCIDR are merged in memory, an IPAddress doesn't come from a specific ServiceCIDR object, but "any ServiceCIDR that includes that IP"
162
162
163
163
The new allocator logic can be used by other APIs, like Gateway API.
164
164
@@ -252,13 +252,13 @@ multiple apiservers.
252
252
253
253
The new allocation mode requires:
254
254
255
-
- 2 new API objects ServiceCIDRConfig and IPAddress in networking.k8s.io/v1alpha1, see <https://groups.google.com/g/kubernetes-sig-api-machinery/c/S0KuN_PJYXY/m/573BLOo4EAAJ>. Both will be protected with finalizers.
255
+
- 2 new API objects ServiceCIDR and IPAddress in networking.k8s.io/v1alpha1, see <https://groups.google.com/g/kubernetes-sig-api-machinery/c/S0KuN_PJYXY/m/573BLOo4EAAJ>. Both will be protected with finalizers.
256
256
- 1 new allocator implementing current `allocator.Interface`, that runs in each apiserver, and uses the new objects to allocate IPs for Services.
257
-
- 1 new controller that participates on the ServiceCIDRConfig deletion, the guarantee that each IPAddresses has
258
-
a ServiceCIDRConfig associated. It also handles the special case for the `default`ServiceCIDRConfig.
257
+
- 1 new controller that participates on the ServiceCIDR deletion, the guarantee that each IPAddresses has
258
+
a ServiceCIDR associated. It also handles the special case for the `default`ServiceCIDR.
259
259
- 1 new controller that participates in the IPAddress deletion, that guarantees that each Service IP
260
260
allocated has its corresponding IPAddress object, recreating it if necessary.
261
-
- 1 new controller that handles the bootstrap process and the default ServiceCIDRConfig.
261
+
- 1 new controller that handles the bootstrap process and the default ServiceCIDR.
For any Service and IPAddress that doesn't belong to a ServiceCIDRConfig the controller will raise an event
386
+
For any Service and IPAddress that doesn't belong to a ServiceCIDR the controller will raise an event
387
387
informing the user, keeping the previous behavior
388
388
389
389
```go
390
390
// cluster IP is out of range
391
391
c.recorder.Eventf(&svc, nil, v1.EventTypeWarning, "ClusterIPOutOfRange", "ClusterIPAllocation", "Cluster IP [%v]:%s is not within the service CIDR %s; please recreate service",
392
392
```
393
393
394
-
- Valid Service and ServiceCIDRConfig but not IPAddress
394
+
- Valid Service and ServiceCIDR but not IPAddress
395
395
396
-
It can happen that an user forcefully delete an IPAddress, in this case, the controller will regenerate the IPAddress, as long as a valid ServiceCIDRConfig covers it.
396
+
It can happen that an user forcefully delete an IPAddress, in this case, the controller will regenerate the IPAddress, as long as a valid ServiceCIDR covers it.
397
397
398
398
During this time, there is a chance that an apiserver tries to allocate this IPAddress, with a possible situation where
399
399
2 Services has the same IPAddress. In order to avoid it, the Allocator will not delete an IP from its local cache
400
400
until it verifies that the consumer associated to that IP has been deleted too.
401
401
402
-
- Valid IPAddress and ServiceCIDRConfig but no Service
402
+
- Valid IPAddress and ServiceCIDR but no Service
403
403
404
404
The IPAddress will be deleted and an event generated if the controller determines that the IPAddress
405
405
is orphan [(see Allocator section)](#Allocator)
@@ -444,24 +444,24 @@ correct, and will start sending events to warn the user that it has to fix/recre
444
444
#### API
445
445
446
446
```go
447
-
//ServiceCIDRConfig defines a range of IPs using CIDR format (192.168.0.0/24 or 2001:db2::0/64).
448
-
typeServiceCIDRConfigstruct {
447
+
//ServiceCIDR defines a range of IPs using CIDR format (192.168.0.0/24 or 2001:db2::0/64).
448
+
typeServiceCIDRstruct {
449
449
metav1.TypeMeta`json:",inline"`
450
450
metav1.ObjectMeta`json:"metadata,omitempty"`
451
451
452
-
SpecServiceCIDRConfigSpec`json:"spec,omitempty"`
452
+
SpecServiceCIDRSpec`json:"spec,omitempty"`
453
453
}
454
454
455
455
456
-
//ServiceCIDRConfigSpec describe how the ServiceCIDRConfig's specification looks like.
457
-
typeServiceCIDRConfigSpecstruct {
456
+
//ServiceCIDRSpec describe how the ServiceCIDR's specification looks like.
457
+
typeServiceCIDRSpecstruct {
458
458
// IPv4 is an IPv4 block in CIDR notation "10.0.0.0/8"
459
459
IPv4string`json:"ipv4"`
460
460
// IPv6 is an IPv6 block in CIDR notation "fd12:3456:789a:1::/64"
461
461
IPv6string`json:"ipv6"`
462
462
}
463
463
464
-
// IPAddress represents an IP used by Kubernetes associated to an ServiceCIDRConfig.
464
+
// IPAddress represents an IP used by Kubernetes associated to an ServiceCIDR.
465
465
// The name of the object is the IP address in canonical format.
466
466
typeIPAddressstruct {
467
467
metav1.TypeMeta`json:",inline"`
@@ -514,10 +514,10 @@ type Interface interface {
514
514
}
515
515
```
516
516
517
-
This allocator will have an informer watching Services, ServiceCIDRConfigs and IPAddresses, so it can have locally the
517
+
This allocator will have an informer watching Services, ServiceCIDRs and IPAddresses, so it can have locally the
518
518
information needed to assign new IP addresses to Services.
519
519
520
-
IPAddresses can only be allocated from ServiceCIDRConfigs that are available and not being deleted.
520
+
IPAddresses can only be allocated from ServiceCIDRs that are available and not being deleted.
521
521
522
522
The uniqueness of an IPAddress is guaranteed by the apiserver, since trying to create an IP
523
523
address that already exist will fail.
@@ -533,7 +533,7 @@ This is a very core and critical change, it has to be thoroughly tested on diffe
533
533
534
534
API objects must have unit tests for defaulting and validation and integration tests exercising the
535
535
different operations and fields permutations, with both positive and negative cases: Special
536
-
attention to cross-reference validation problems, like create IPAddress reference wrong ServiceCIDRConfig or
536
+
attention to cross-reference validation problems, like create IPAddress reference wrong ServiceCIDR or
537
537
invalid or missing
538
538
539
539
Controllers must have unit tests and integration tests covering all possible race conditions.
@@ -616,9 +616,9 @@ We expect no non-infra related flakes in the last month as a GA graduation crite
616
616
- API stability, no changes on new types and behaviors:
617
617
- ServiceCIDR immutability
618
618
- default IPFamily
619
-
- two or one IP families per ServiceCIDRConfig
619
+
- two or one IP families per ServiceCIDR
620
620
- Gather feedback from developers and users
621
-
- Document and add more complex testing scenarios: scaling out ServiceCIDRConfigs, ...
621
+
- Document and add more complex testing scenarios: scaling out ServiceCIDRs, ...
622
622
- Additional tests are in Testgrid and linked in KEP
623
623
- Scale test to O(10K) services and O(1K) ranges
624
624
- Improve performance on the allocation logic, O(1) for allocating a free address
@@ -664,7 +664,7 @@ Example:
664
664
665
665
- flags set to 10.0.0.0/20
666
666
- upgrade to 1.25 with alpha gate
667
-
- apiserver create a default ServiceCIDRConfig object for 10.0.0.0/20
667
+
- apiserver create a default ServiceCIDR object for 10.0.0.0/20
668
668
- user creates a new ServiceCIDR for 192.168.1.0/24
669
669
- create a Service which gets 192.168.1.1
670
670
- rollback or disable the gate
@@ -678,7 +678,7 @@ Example:
678
678
###### How can this feature be enabled / disabled in a live cluster?
679
679
680
680
-[x] Feature gate (also fill in values in `kep.yaml`)
681
-
- Feature gate name: ServiceCIDRConfig
681
+
- Feature gate name: ServiceCIDR
682
682
- Components depending on the feature gate: kube-apiserver, kube-controller-manager
683
683
684
684
###### Does enabling the feature change any default behavior?
@@ -891,7 +891,7 @@ Think about adding additional work or introducing new steps in between
891
891
892
892
###### Will enabling / using this feature result in non-negligible increase of resource usage (CPU, RAM, disk, IO, ...) in any components?
893
893
894
-
The apiservers will increase the memory usage because they have to keep a local informer with the new objects ServiceCIDRConfig and IPAddress.
894
+
The apiservers will increase the memory usage because they have to keep a local informer with the new objects ServiceCIDR and IPAddress.
0 commit comments