GEP-91: Address connection coalescing security issue - API updates

[kl52752]

What type of PR is this?
/kind gep

What this PR does / why we need it:
Implements API changes for GEP-91

Which issue(s) this PR fixes:

Fixes 3567
Relates to: #3760 (comment)

Does this PR introduce a user-facing change?:

NONE

[k8s-ci-robot]

Hi @kl52752. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[kl52752]

/assign @robscott @arkodg @youngnick @shaneutt

[youngnick]

Small clarification about pointer-to-string, but aside from that, LGTM.

[youngnick]

/ok-to-test
/approve

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: kl52752, youngnick

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [youngnick]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[kl52752]

/retest

[kl52752]

/retest

[robscott]

Thanks @kl52752!

[robscott]

🤔 This field name feels a little funny to me. Something about configs as a plural word doesn't seem quite right. Some alternatives that may not be better:

1. tlsByPort
2. tlsPerPort
3. tls
4. frontendTLS

Since even from the outset port is optional, 1 or 2 probably aren't great. Since there's at least a theoretical world where we add something TLS related here that isn't tied to frontend TLS, we could regret 4 in the future. So maybe 3 is the least bad, although even that adds some confusion as that tls field would have no overlap with listener.tls. Open to other ideas here.

/cc @arkodg @youngnick

[arkodg]

if there is no strict rule for pluralizing / s slice fields, then +1 for tls

[youngnick]

I do like the idea of tlsByPort or tlsPerPort, but I think that might also be confusing with the defaulting behavior?

I think that tls is an acceptable alternative though.

[kl52752]

I find the tls name for an array a bit confusing (specially the case when you iterate over tls). If we want "default" configuration to be explicit (see comment) we could consider creating a struct with default configuration (no port at all) and array with per ports override. This way it will be explicit, readable and not at all confusing :)

type GatewayTLSConfig struct {
defaultTLS *FrontendTLSValidation
tlsPerPort []TLSConfig
}

and in GatewaySpec we can name this field as tls

[youngnick]

I added a different proposal above, but this could also work.

[kl52752]

I personally prefer mine solution than Nicks. I like it that default configuration is extracted from per ports override and now when default configuration is required it makes much sense.
I need to add more validations for per port overrides like uniquest etc but port = 0 will be forbidden.

@shaneutt @robscott which version do you prefer?

[youngnick]

After some reflection, I think that @kl52752's option here is better, agreed. Separating the default config into its own section lets us say "the default config applies to all listeners with matching ports that terminate TLS, unless there is a matching config in the tlsPerPort section. In that case only the config in the tlsPerPort section applies."

[robscott]

I'm fine with that approach as well, thanks @kl52752!

[shaneutt]

👍

[kl52752]

/retest

[PT-GD]

@kl52752 I am looking for a solution where the same certificate is on two distinct listeners on a gateway. This is necessary because the Gateway API allows a single listener per gateway and the hostname for a listener can be either an FQDN or a wildcard FQDN. The behavior of wildcard certificates in TLS is that they match the apex (FQDN) or one label deep of wildcarded names.

Will this GEP address coalescing connections to prevent HTTP/2 connection reuse issues when mutual TLS (client certificates) are not involved and two listeners attached to a gateway share the same server certificate? [Perhaps there should be a test for this, if there isn't one]?

See also https://istio.io/latest/docs/ops/common-problems/network-issues/#404-errors-occur-when-multiple-gateways-configured-with-same-tls-certificate for how this impact users of the Istio controller for the Gateway API.

I think we have consensus now on the approach, so removing my block.

[kl52752]

@kl52752 I am looking for a solution where the same certificate is on two distinct listeners on a gateway. This is necessary because the Gateway API allows a single listener per gateway and the hostname for a listener can be either an FQDN or a wildcard FQDN. The behavior of wildcard certificates in TLS is that they match the apex (FQDN) or one label deep of wildcarded names.

Will this GEP address coalescing connections to prevent HTTP/2 connection reuse issues when mutual TLS (client certificates) are not involved and two listeners attached to a gateway share the same server certificate? [Perhaps there should be a test for this, if there isn't one]?

See also https://istio.io/latest/docs/ops/common-problems/network-issues/#404-errors-occur-when-multiple-gateways-configured-with-same-tls-certificate for how this impact users of the Istio controller for the Gateway API.

hi @PT-GD in this GEP it is possible to attach the same FrontendValidation config (referencing the same ConfigMap) to 2 distinct Listeners in tlsPerPort field. The restriction here is that every port needs unique configuration but not the other way.
If they are listening on different port, you can do this by creating TLSConfig struct with the same FrontendTLSValidation but different port. If they are Listening on the same port only one FrontendTLSValidation is needed:
see the snippet:

apiVersion: gateway.networking.k8s.io/v1beta1
kind: Gateway
metadata:
  name: client-validation-basic
spec:
  gatewayClassName: acme-lb
  tls:
    defaultTLS: 
      caCertificateRefs:
        - kind: ConfigMap
          group: ""
          name: default-cert
        mode: AllowInsecureFallback
    tlsPerPort:
        - port: 443 // this configuration will apply to all Listeners matching this port
          frontendValidation:
            caCertificateRefs:
            - kind: ConfigMap
            group: ""
            name: foo-example-com-ca-cert
        - port: 8443 // If needed you can create second entry referencing the same ConfigMap for another port
          frontendValidation:
            caCertificateRefs:
            - kind: ConfigMap
            group: ""
            name: foo-example-com-ca-cert
            

[robscott]

Thanks @kl52752!

[robscott]

Want to avoid names that stutter (tls.defaultTLS -> tls.default)

Suggested change

	DefaultTLS FrontendTLSValidation `json:"defaultTLS"`
	Default FrontendTLSValidation `json:"default"`

[robscott]

Also recommend an intermediate struct here so we leave room for other forms of TLS config at this default level. So tls.default.frontendValidation.mode instead of tls.default.mode.

[robscott]

Similar comment to above.

Suggested change

	TLSPerPort []TLSConfig `json:"tlsPerPort,omitempty"`
	PerPort []TLSConfig `json:"perPort,omitempty"`

[robscott]

Recommend something like this to make it clear that this is per-port.

Suggested change

	type TLSConfig struct {
	type TLSPortConfig struct {