You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
* Mixed Protocol LB KEP updates
Migrated to the new format
Status set to implementable
Some more test plan details added
* fix review comments
* fix review comments
* add line feeds
* update toc
-[ ] kubernetes/enhancements issue in release milestone, which links to KEP (this should be a link to the KEP location in kubernetes/enhancements, not the initial KEP PR) https://github.com/kubernetes/enhancements/issues/1435
70
-
-[ ] KEP approvers have set the KEP status to `implementable`
71
-
-[ ] Design details are appropriately documented
72
-
-[ ] Test plan is in place, giving consideration to SIG Architecture and SIG Testing input
73
-
-[ ] Graduation criteria is in place
56
+
Items marked with (R) are required *prior to targeting to a milestone / release*.
57
+
58
+
-[ ] (R) Enhancement issue in release milestone, which links to KEP dir in [kubernetes/enhancements] (not the initial KEP PR)
59
+
-[ ] (R) KEP approvers have approved the KEP status as `implementable`
60
+
-[ ] (R) Design details are appropriately documented
61
+
-[ ] (R) Test plan is in place, giving consideration to SIG Architecture and SIG Testing input
62
+
-[ ] (R) Graduation criteria is in place
63
+
-[ ] (R) Production readiness review completed
64
+
-[ ] Production readiness review approved
74
65
-[ ] "Implementation History" section is up-to-date for milestone
75
66
-[ ] User-facing documentation has been created in [kubernetes/website], for publication to [kubernetes.io]
76
-
-[ ] Supporting documentation e.g., additional design documents, links to mailing list discussions/SIG meetings, relevant PRs/issues, release notes
67
+
-[ ] Supporting documentation—e.g., additional design documents, links to mailing list discussions/SIG meetings, relevant PRs/issues, release notes
This feature enables the creation of a LoadBalancer Service that has different port definitions with different protocols.
81
78
79
+
82
80
## Motivation
83
81
84
82
The ultimate goal of this feature is to support users that want to expose their applications via a single IP address but different L4 protocols with a cloud provided load-balancer.
@@ -99,6 +97,7 @@ The goals of this KEP are:
99
97
100
98
### Non-Goals
101
99
100
+
N/A
102
101
103
102
## Proposal
104
103
@@ -267,7 +266,6 @@ A user can ask for an internal Load Balancer via a K8s Service definition that
267
266
268
267
Summary: Tencent Cloud CPI and LBs seem to support mixed protocol Services, and the pricing is not based on the number of protocols per Service.
269
268
270
-
271
269
### Risks and Mitigations
272
270
273
271
#### Billing perspective
@@ -307,7 +305,7 @@ Our feature does not introduce new values or new fields. It enables the usage of
307
305
- Oracle: no risk. The CPI and LB already supports mixed protocols. The same situation like in the case of Alibaba.
308
306
- Tencent Cloud: no risk. The CPI and LB already supports mixed protocols. The same situation like in the case of Alibaba.
309
307
310
-
As stated above we must implement a feature gate based phased introduction for this feature because of its effects. See the `Proposed solution` part for details in this document below.
308
+
As stated above we must implement a feature gate based phased introduction for this feature because of its effects. See the `The selected solution for the option control` part for details in this document below.
311
309
312
310
## Design Details
313
311
@@ -475,9 +473,14 @@ In the long term:
475
473
476
474
The kube-proxy should use the port status information from `Service.status.loadBalancer.ingress` in order not to allow traffic to those ports that could not be opened by the load balancer either.
477
475
476
+
478
477
### Test Plan
479
478
480
479
There must be e2e cases that test whether CPI implementations handle Service definitions with mixed protocol configuration on a consistent way. I.e. either the cloud LB is set up properly or the Service is rejected by the CPI implementation.
480
+
The e2e tests shall check that
481
+
- a multi-protocol Service triggers the creation of a multi-protocol cloud load balancer
482
+
Optionally, if the CPI supports that:
483
+
- the CPI sets the new Conditions and or Port Status in the Load Balancer Service after creating the cloud load balancer
481
484
482
485
### Graduation Criteria
483
486
@@ -487,6 +490,21 @@ From CPI implementation perspective thet feature can be graduated to beta, as th
487
490
488
491
Graduating to GA means, that the feature flag checking is removed from the code. It means, that all CPI implementations must be ready to deal with Services with mixed protocol configuration - either rejecting such Services properly or managing the cloud load balancers according to the Service definition.
489
492
493
+
#### Alpha Graduation
494
+
495
+
- Feature is implemented and controller with a feature flag. The feature flag is disabled by default.
496
+
497
+
#### Alpha -> Beta Graduation
498
+
499
+
- We shall have test results or real life usage results from the affected clouds that prove that the enabling of this feature does not break their load balancer management and functionality.
500
+
501
+
#### Beta -> GA Graduation
502
+
503
+
TBD
504
+
505
+
#### Removing a Deprecated Flag
506
+
507
+
TBD
490
508
491
509
### Upgrade / Downgrade Strategy
492
510
@@ -504,20 +522,143 @@ Version skew is possible among the following components in this case: K8s API se
504
522
505
523
Once this feature is implemented in the API server there is a chance that the CPI implementation has to deal with load balancer Services with mixed protocol configuration anytime, even if the API server is downgraded later. The CPI implementation shall be prepared for this, i.e. the CPI implementation cannot expect anymore that the API Server (or any other component, like the Service Controller) filters out such Service definitions. If the CPI implementation wants to have such filtering it has to implement that on its own. In this case the CPI implementation shall be upgraded before the feature is enabled on the API server. In case of a rollback of the API Server such Services can still exist in the cluster, so the CPI implementation should not be downgraded to a version that does not implement that filtering. This is the reason why the CPI implementations shall be updated (if necessary) to be able to deal with such Service definitions already in the first release of this feature.
506
524
525
+
## Production Readiness Review Questionnaire
526
+
527
+
### Feature Enablement and Rollback
528
+
529
+
_This section must be completed when targeting alpha to a release._
530
+
531
+
* **How can this feature be enabled / disabled in a live cluster?**
532
+
- [x] Feature gate (also fill in values in `kep.yaml`)
533
+
- Feature gate name: MixedProtocolLBSVC
534
+
- Components depending on the feature gate: Kubernetes API Server
535
+
536
+
* **Does enabling the feature change any default behavior?**
537
+
538
+
When the feature is enabled the Services with mixed protocols are not rejected anymore by the Kuber API server, and it is up to the CPI to handle those.
539
+
Please see the analysis at `API change and upgrade/downgrade situations`
540
+
541
+
* **Can the feature be disabled once it has been enabled (i.e. can we roll back
542
+
the enablement)?**
543
+
544
+
Yes.
545
+
546
+
* **What happens if we reenable the feature if it was previously rolled back?**
547
+
548
+
Nothing serious, as this feature removes a restriction in the API Server. I.e. this direction does not introduce problems.
549
+
550
+
* **Are there any tests for feature enablement/disablement?**
551
+
552
+
TBD
553
+
554
+
### Rollout, Upgrade and Rollback Planning
555
+
556
+
_This section must be completed when targeting beta graduation to a release._
557
+
558
+
* **How can a rollout fail? Can it impact already running workloads?**
559
+
560
+
TBD
561
+
562
+
* **What specific metrics should inform a rollback?**
563
+
564
+
TBD
565
+
566
+
* **Were upgrade and rollback tested? Was the upgrade->downgrade->upgrade path tested?**
567
+
568
+
TBD
569
+
570
+
### Monitoring Requirements
571
+
572
+
_This section must be completed when targeting beta graduation to a release._
573
+
574
+
* **How can an operator determine if the feature is in use by workloads?**
575
+
576
+
TBD
577
+
578
+
* **What are the SLIs (Service Level Indicators) an operator can use to determine
579
+
the health of the service?**
580
+
- [ ] Metrics
581
+
- Metric name:
582
+
- [Optional] Aggregation method:
583
+
- Components exposing the metric:
584
+
- [ ] Other (treat as last resort)
585
+
- Details:
586
+
587
+
* **What are the reasonable SLOs (Service Level Objectives) for the above SLIs?**
588
+
589
+
TBD
590
+
591
+
* **Are there any missing metrics that would be useful to have to improve observability
592
+
of this feature?**
593
+
594
+
TBD
595
+
596
+
### Dependencies
597
+
598
+
_This section must be completed when targeting beta graduation to a release._
599
+
600
+
* **Does this feature depend on any specific services running in the cluster?**
601
+
CPIs shall be prepared to handle Service definitions with mixed protocols. Please see the analysis above.
602
+
603
+
604
+
### Scalability
605
+
606
+
* **Will enabling / using this feature result in any new API calls?**
607
+
608
+
If a CPI supports the management of the new Conditions and PortStatus in the LoadBalancer Service the managemenof of those fileds will mean additional traffic on the API
609
+
610
+
* **Will enabling / using this feature result in introducing new API types?**
611
+
612
+
No
613
+
614
+
* **Will enabling / using this feature result in any new calls to the cloud
615
+
provider?**
616
+
617
+
If the cloud provider requires more calls to add ports/listeners with different protocols to a load balancer then this feature introduces additional calls
618
+
619
+
* **Will enabling / using this feature result in increasing size or count of
620
+
the existing API objects?**
621
+
622
+
Yes. As detailed above, the Status of the Service is planned to be extended with new Conditions and PortStatus
623
+
624
+
* **Will enabling / using this feature result in increasing time taken by any
625
+
operations covered by [existing SLIs/SLOs]?**
626
+
627
+
The setup of more ports/listeners with different protocols may take more time, depending on how the CPI and the cloud provider API is implemented
628
+
629
+
* **Will enabling / using this feature result in non-negligible increase of
630
+
resource usage (CPU, RAM, disk, IO, ...) in any components?**
631
+
632
+
Not expected.
633
+
634
+
### Troubleshooting
635
+
636
+
* **How does this feature react if the API server and/or etcd is unavailable?**
637
+
638
+
* **What are other known failure modes?**
639
+
640
+
TBD
641
+
642
+
* **What steps should be taken if SLOs are not being met to determine the problem?**
Major milestones in the life cycle of a KEP should be tracked in `Implementation History`.
510
-
Major milestones might include
651
+
- the `Proposal` section being merged, signaling agreement on a proposed design: 14th July 2020
652
+
653
+
## Drawbacks
654
+
655
+
TBD
511
656
512
-
- the `Summary` and `Motivation` sections being merged signaling SIG acceptance
513
-
- the `Proposal` section being merged signaling agreement on a proposed design
514
-
- the date implementation started
515
-
- the first Kubernetes release where an initial version of the KEP was available
516
-
- the version of Kubernetes where the KEP graduated to general availability
517
-
- when the KEP was retired or superseded
657
+
## Alternatives
518
658
519
-
## Drawbacks [optional]
659
+
No known alternatives, beside those analysed above.
520
660
521
-
## Alternatives [optional]
661
+
## Infrastructure Needed (Optional)
522
662
523
-
## Infrastructure Needed [optional]
663
+
The Cloud Load Balancer shall support the setup of different protocols on the same IP address in order to utilize the benefits of this feature.
664
+
The CPI implementation shall either indicate problems with the Service in the new status fields, or the CPI shall be able to setup the cloud load balancer with the different protocols.
0 commit comments