Skip to content

Commit a0465bd

Browse files
0xch4z0xch4z
authored
Merge pull request #86 from Charliekenney23/feat/nodebal-per-port-config
allow configuring proxy-protocol per port
2 parents 62b517e + 3420b69 commit a0465bd

File tree

6 files changed

+209
-47
lines changed

6 files changed

+209
-47
lines changed

‎README.md‎

Lines changed: 20 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -44,8 +44,8 @@ Annotation (Suffix) | Values | Default | Description
4444
---|---|---|---
4545
`throttle` | `0`-`20` (`0` to disable) | `20` | Client Connection Throttle, which limits the number of subsequent new connections per second from the same client IP
4646
`default-protocol` | `tcp`, `http`, `https` | `tcp` | This annotation is used to specify the default protocol for Linode NodeBalancer.
47-
`proxy-protocol` | `none`, `v1`, `v2` | `none` | Specifies whether to use a version of Proxy Protocol on the underlying NodeBalancer
48-
`port-*` | json (e.g. `{ "tls-secret-name": "prod-app-tls", "protocol": "https"}`) | | Specifies the secret and protocol for a port corresponding secrets. The secret type should be `kubernetes.io/tls`. `*` is the port being configured, e.g. `linode-loadbalancer-port-443`
47+
`default-proxy-protocol` | `none`, `v1`, `v2` | `none` | Specifies whether to use a version of Proxy Protocol on the underlying NodeBalancer.
48+
`port-*` | json (e.g. `{ "tls-secret-name": "prod-app-tls", "protocol": "https", "proxy-protocol": "v2"}`) | | Specifies port specific NodeBalancer configuration. See [Port Specific Configuration](#port-specific-configuration). `*` is the port being configured, e.g. `linode-loadbalancer-port-443`
4949
`check-type` | `none`, `connection`, `http`, `http_body` | | The type of health check to perform against back-ends to ensure they are serving requests
5050
`check-path` | string | | The URL path to check on each back-end during health checks
5151
`check-body` | string | | Text which must be present in the response body to pass the NodeBalancer health check
@@ -58,17 +58,28 @@ Annotation (Suffix) | Values | Default | Description
5858

5959
#### Deprecated Annotations
6060

61-
These annotations are deprecated, and will be removed Q3 2020.
61+
These annotations are deprecated, and will be removed in a future release.
6262

63-
Annotation (Suffix) | Values | Default | Description
64-
---|---|---|---
65-
`protocol` | `tcp`, `http`, `https` | `tcp` | This annotation is used to specify the default protocol for Linode NodeBalancer. For ports specified in the `linode-loadbalancer-tls-ports` annotation, this protocol is overwritten to `https`
66-
`tls` | json array (e.g. `[ { "tls-secret-name": "prod-app-tls", "port": 443}, {"tls-secret-name": "dev-app-tls", "port": 8443} ]`) | | Specifies TLS ports with their corresponding secrets, the secret type should be `kubernetes.io/tls
63+
Annotation (Suffix) | Values | Default | Description | Scheduled Removal
64+
---|---|---|---|---
65+
`protocol` | `tcp`, `http`, `https` | `tcp` | This annotation is used to specify the default protocol for Linode NodeBalancer. For ports specified in the `linode-loadbalancer-tls-ports` annotation, this protocol is overwritten to `https` | Q4 2020
66+
`proxy-protcol` | `none`, `v1`, `v2` | `none` | Specifies whether to use a version of Proxy Protocol on the underlying NodeBalancer | Q4 2021
67+
`tls` | json array (e.g. `[ { "tls-secret-name": "prod-app-tls", "port": 443}, {"tls-secret-name": "dev-app-tls", "port": 8443} ]`) | | Specifies TLS ports with their corresponding secrets, the secret type should be `kubernetes.io/tls | Q4 2020
6768

6869
#### Annotation bool values
6970

7071
For annotations with bool value types, `"1"`, `"t"`, `"T"`, `"True"`, `"true"` and `"True"` are valid string representations of `true`. Any other values will be interpreted as false. For more details, see [strconv.ParseBool](https://golang.org/pkg/strconv/#ParseBool).
7172

73+
#### Port Specific Configuration
74+
75+
These configuration options can be specified via the `port-*` annotation, encoded in JSON.
76+
77+
Key | Values | Default | Description
78+
---|---|---|---
79+
`protocol` | `tcp`, `http`, `https` | `tcp` | Specifies protocol of the NodeBalancer port. Overwrites `default-protocol`.
80+
`proxy-protocol` | `none`, `v1`, `v2` | `none` | Specifies whether to use a version of Proxy Protocol on the underlying NodeBalancer. Overwrites `default-proxy-protocol`.
81+
`tls-secret-name` | string | | Specifies a secret to use for TLS. The secret type should be `kubernetes.io/tls`.
82+
7283
#### Example usage
7384

7485
```yaml
@@ -128,10 +139,10 @@ spec:
128139

129140
See more in the [examples directory](examples)
130141

131-
## Why `stickiness` and `algorithm` annotations don't exit
142+
## Why `stickiness` and `algorithm` annotations don't exist
132143

133144
As kube-proxy will simply double-hop the traffic to a random backend Pod anyway, it doesn't matter which backend Node traffic is forwarded-to for the sake of session stickiness.
134-
So these annotations are not necessary to implement session stickiness.
145+
These annotations are not necessary to implement session stickiness, as kube-proxy will simply double-hop the packets to a random backend Pod. It would not make a difference to set a backend Node that would receive the network traffic in an attempt to set session stickiness.
135146

136147
## How to use sessionAffinity
137148

‎cloud/linode/loadbalancers.go‎

Lines changed: 30 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -24,9 +24,9 @@ import (
2424
const (
2525
// annLinodeDefaultProtocol is the annotation used to specify the default protocol
2626
// for Linode load balancers. Options are tcp, http and https. Defaults to tcp.
27-
annLinodeDefaultProtocol = "service.beta.kubernetes.io/linode-loadbalancer-default-protocol"
28-
annLinodePortConfigPrefix = "service.beta.kubernetes.io/linode-loadbalancer-port-"
29-
annLinodeProxyProtocol = "service.beta.kubernetes.io/linode-loadbalancer-proxy-protocol"
27+
annLinodeDefaultProtocol = "service.beta.kubernetes.io/linode-loadbalancer-default-protocol"
28+
annLinodePortConfigPrefix = "service.beta.kubernetes.io/linode-loadbalancer-port-"
29+
annLinodeDefaultProxyProtocol = "service.beta.kubernetes.io/linode-loadbalancer-default-proxy-protocol"
3030

3131
annLinodeCheckPath = "service.beta.kubernetes.io/linode-loadbalancer-check-path"
3232
annLinodeCheckBody = "service.beta.kubernetes.io/linode-loadbalancer-check-body"
@@ -68,11 +68,13 @@ type loadbalancers struct {
6868
type portConfigAnnotation struct {
6969
TLSSecretName string `json:"tls-secret-name"`
7070
Protocol string `json:"protocol"`
71+
ProxyProtocol string `json:"proxy-protocol"`
7172
}
7273

7374
type portConfig struct {
7475
TLSSecretName string
7576
Protocol linodego.ConfigProtocol
77+
ProxyProtocol linodego.ConfigProxyProtocol
7678
Port int
7779
}
7880

@@ -478,9 +480,10 @@ func (l *loadbalancers) buildNodeBalancerConfig(service *v1.Service, port int) (
478480
}
479481

480482
config := linodego.NodeBalancerConfig{
481-
Port: port,
482-
Protocol: portConfig.Protocol,
483-
Check: health,
483+
Port: port,
484+
Protocol: portConfig.Protocol,
485+
ProxyProtocol: portConfig.ProxyProtocol,
486+
Check: health,
484487
}
485488

486489
if health == linodego.CheckHTTP || health == linodego.CheckHTTPBody {
@@ -530,17 +533,6 @@ func (l *loadbalancers) buildNodeBalancerConfig(service *v1.Service, port int) (
530533
}
531534
config.CheckPassive = checkPassive
532535

533-
proxyProtocol := linodego.ProxyProtocolNone
534-
if pp, ok := service.Annotations[annLinodeProxyProtocol]; ok {
535-
switch linodego.ConfigProxyProtocol(pp) {
536-
case linodego.ProxyProtocolNone, linodego.ProxyProtocolV1, linodego.ProxyProtocolV2:
537-
proxyProtocol = linodego.ConfigProxyProtocol(pp)
538-
default:
539-
return config, fmt.Errorf("invalid NodeBalancer proxy protocol value '%s'", pp)
540-
}
541-
}
542-
config.ProxyProtocol = proxyProtocol
543-
544536
if portConfig.Protocol == linodego.ProtocolHTTPS {
545537
if err = l.addTLSCert(service, &config, portConfig); err != nil {
546538
return config, err
@@ -643,15 +635,35 @@ func getPortConfig(service *v1.Service, port int) (portConfig, error) {
643635
protocol = "tcp"
644636
}
645637
}
646-
647638
protocol = strings.ToLower(protocol)
648639

640+
proxyProtocol := portConfigAnnotation.ProxyProtocol
641+
if proxyProtocol == "" {
642+
var ok bool
643+
for _, ann := range []string{annLinodeDefaultProxyProtocol, annLinodeProxyProtocol} {
644+
proxyProtocol, ok = service.Annotations[ann]
645+
if ok {
646+
break
647+
} else {
648+
proxyProtocol = string(linodego.ProxyProtocolNone)
649+
}
650+
}
651+
}
652+
649653
if protocol != "tcp" && protocol != "http" && protocol != "https" {
650654
return portConfig, fmt.Errorf("invalid protocol: %q specified", protocol)
651655
}
652656

657+
switch proxyProtocol {
658+
case string(linodego.ProxyProtocolNone), string(linodego.ProxyProtocolV1), string(linodego.ProxyProtocolV2):
659+
break
660+
default:
661+
return portConfig, fmt.Errorf("invalid NodeBalancer proxy protocol value '%s'", proxyProtocol)
662+
}
663+
653664
portConfig.Port = port
654665
portConfig.Protocol = linodego.ConfigProtocol(protocol)
666+
portConfig.ProxyProtocol = linodego.ConfigProxyProtocol(proxyProtocol)
655667
portConfig.TLSSecretName = portConfigAnnotation.TLSSecretName
656668

657669
return portConfig, nil

‎cloud/linode/loadbalancers_deprecated.go‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@ import (
99
const (
1010
annLinodeProtocolDeprecated = "service.beta.kubernetes.io/linode-loadbalancer-protocol"
1111
annLinodeLoadBalancerTLSDeprecated = "service.beta.kubernetes.io/linode-loadbalancer-tls"
12+
annLinodeProxyProtocol = "service.beta.kubernetes.io/linode-loadbalancer-proxy-protocol"
1213
)
1314

1415
type tlsAnnotationDeprecated struct {

‎cloud/linode/loadbalancers_test.go‎

Lines changed: 60 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -573,7 +573,7 @@ func testUpdateLoadBalancerAddProxyProtocol(t *testing.T, client *linodego.Clien
573573

574574
svc.Status.LoadBalancer = *makeLoadBalancerStatus(nodeBalancer)
575575
svc.ObjectMeta.SetAnnotations(map[string]string{
576-
annLinodeProxyProtocol: string(tc.proxyProtocolConfig),
576+
annLinodeDefaultProxyProtocol: string(tc.proxyProtocolConfig),
577577
})
578578

579579
stubService(fakeClientset, svc)
@@ -771,6 +771,60 @@ func Test_getPortConfig(t *testing.T) {
771771
expectedPortConfig portConfig
772772
err error
773773
}{
774+
{
775+
"default no proxy protocol specified",
776+
&v1.Service{
777+
ObjectMeta: metav1.ObjectMeta{
778+
Name: randString(10),
779+
UID: "abc123",
780+
},
781+
},
782+
portConfig{Port: 443, Protocol: "tcp", ProxyProtocol: linodego.ProxyProtocolNone},
783+
nil,
784+
},
785+
{
786+
"default proxy protocol specified",
787+
&v1.Service{
788+
ObjectMeta: metav1.ObjectMeta{
789+
Name: randString(10),
790+
UID: "abc123",
791+
Annotations: map[string]string{
792+
annLinodeDefaultProxyProtocol: string(linodego.ProxyProtocolV2),
793+
},
794+
},
795+
},
796+
portConfig{Port: 443, Protocol: "tcp", ProxyProtocol: linodego.ProxyProtocolV2},
797+
nil,
798+
},
799+
{
800+
"port specific proxy protocol specified",
801+
&v1.Service{
802+
ObjectMeta: metav1.ObjectMeta{
803+
Name: randString(10),
804+
UID: "abc123",
805+
Annotations: map[string]string{
806+
annLinodeDefaultProxyProtocol: string(linodego.ProxyProtocolV2),
807+
annLinodePortConfigPrefix + "443": fmt.Sprintf(`{"proxy-protocol": "%s"}`, linodego.ProxyProtocolV1),
808+
},
809+
},
810+
},
811+
portConfig{Port: 443, Protocol: "tcp", ProxyProtocol: linodego.ProxyProtocolV1},
812+
nil,
813+
},
814+
{
815+
"default invalid proxy protocol",
816+
&v1.Service{
817+
ObjectMeta: metav1.ObjectMeta{
818+
Name: randString(10),
819+
UID: "abc123",
820+
Annotations: map[string]string{
821+
annLinodeDefaultProxyProtocol: "invalid",
822+
},
823+
},
824+
},
825+
portConfig{},
826+
fmt.Errorf("invalid NodeBalancer proxy protocol value '%s'", "invalid"),
827+
},
774828
{
775829
"default no protocol specified",
776830
&v1.Service{
@@ -779,7 +833,7 @@ func Test_getPortConfig(t *testing.T) {
779833
UID: "abc123",
780834
},
781835
},
782-
portConfig{Port: 443, Protocol: "tcp"},
836+
portConfig{Port: 443, Protocol: "tcp", ProxyProtocol: linodego.ProxyProtocolNone},
783837

784838
nil,
785839
},
@@ -794,7 +848,7 @@ func Test_getPortConfig(t *testing.T) {
794848
},
795849
},
796850
},
797-
portConfig{Port: 443, Protocol: "tcp"},
851+
portConfig{Port: 443, Protocol: "tcp", ProxyProtocol: linodego.ProxyProtocolNone},
798852
nil,
799853
},
800854
{
@@ -808,7 +862,7 @@ func Test_getPortConfig(t *testing.T) {
808862
},
809863
},
810864
},
811-
portConfig{Port: 443, Protocol: "http"},
865+
portConfig{Port: 443, Protocol: "http", ProxyProtocol: linodego.ProxyProtocolNone},
812866
nil,
813867
},
814868
{
@@ -837,7 +891,7 @@ func Test_getPortConfig(t *testing.T) {
837891
},
838892
},
839893
},
840-
portConfig{Port: 443, Protocol: "http"},
894+
portConfig{Port: 443, Protocol: "http", ProxyProtocol: linodego.ProxyProtocolNone},
841895
nil,
842896
},
843897
{
@@ -851,7 +905,7 @@ func Test_getPortConfig(t *testing.T) {
851905
},
852906
},
853907
},
854-
portConfig{Port: 443, Protocol: "http"},
908+
portConfig{Port: 443, Protocol: "http", ProxyProtocol: linodego.ProxyProtocolNone},
855909
nil,
856910
},
857911
{

0 commit comments

Comments
 (0)