`linkerd install --crds` messes up existing Gateway API installation

[kflynn]

What is the issue?

If you do this:

kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.0.0/experimental-install.yaml
linkerd install --crds | kubectl apply -f -

the linkerd install ends up removing httproutes.gateway.networking.k8s.io v1. This is a problem for things that are trying to use the v1 Gateway API CRDs.

How can it be reproduced?

See above.

Logs, error output, etc

(base) …/gamma-workshop [☸ k3d-gateway-api-workshop] on  main [!?] took 14s
:; kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.0.0/experimental-install.yaml
customresourcedefinition.apiextensions.k8s.io/backendtlspolicies.gateway.networking.k8s.io created
customresourcedefinition.apiextensions.k8s.io/gatewayclasses.gateway.networking.k8s.io created
customresourcedefinition.apiextensions.k8s.io/gateways.gateway.networking.k8s.io created
customresourcedefinition.apiextensions.k8s.io/grpcroutes.gateway.networking.k8s.io created
customresourcedefinition.apiextensions.k8s.io/httproutes.gateway.networking.k8s.io created
customresourcedefinition.apiextensions.k8s.io/referencegrants.gateway.networking.k8s.io created
customresourcedefinition.apiextensions.k8s.io/tcproutes.gateway.networking.k8s.io created
customresourcedefinition.apiextensions.k8s.io/tlsroutes.gateway.networking.k8s.io created
customresourcedefinition.apiextensions.k8s.io/udproutes.gateway.networking.k8s.io created
(base) …/gamma-workshop [☸ k3d-gateway-api-workshop] on  main [!?] took 920ms
:; k get crd httproutes.gateway.networking.k8s.io -o yaml | grep 'name: v'
    name: v1
    name: v1beta1
(base) …/gamma-workshop [☸ k3d-gateway-api-workshop] on  main [!?] took 1s
:; linkerd install --crds | kubectl apply -f -

Rendering Linkerd CRDs...
Next, run `linkerd install | kubectl apply -f -` to install the control plane.

customresourcedefinition.apiextensions.k8s.io/authorizationpolicies.policy.linkerd.io created
customresourcedefinition.apiextensions.k8s.io/httproutes.policy.linkerd.io created
customresourcedefinition.apiextensions.k8s.io/meshtlsauthentications.policy.linkerd.io created
customresourcedefinition.apiextensions.k8s.io/networkauthentications.policy.linkerd.io created
customresourcedefinition.apiextensions.k8s.io/serverauthorizations.policy.linkerd.io created
customresourcedefinition.apiextensions.k8s.io/servers.policy.linkerd.io created
customresourcedefinition.apiextensions.k8s.io/serviceprofiles.linkerd.io created
customresourcedefinition.apiextensions.k8s.io/httproutes.gateway.networking.k8s.io configured
customresourcedefinition.apiextensions.k8s.io/externalworkloads.workload.linkerd.io created
(base) …/gamma-workshop [☸ k3d-gateway-api-workshop] on  main [!?] took 504ms
:; k get crd httproutes.gateway.networking.k8s.io -o yaml | grep 'name: v'
    name: v1alpha2
    name: v1beta1

output of linkerd check -o short

This is with edge-24.2.3, but I imagine it extends to other versions.

Environment

Anything using Gateway API.

Possible solution

It's possible to work around this by installing the Gateway API CRDs after LInkerd.

Additional context

No response

Would you like to work on fixing this bug?

None

[kflynn]

Another workaround is linkerd install --crds --set enableHttpRoutes=false, which will prevent Linkerd from overwriting the Gateway API HTTPRoute. That still feels like a workaround, since the default case can mess with you if you're trying to actually use Gateway API.

[wmorgan]

Real-life use case documented in https://matduggan.com/the-time-linkerd-erased-my-load-balancer/. (Sorry Mat!)

Agreed this isn't right. On the docket to be fixed.

[kflynn]

The more I work with Gateway API, Linkerd, and Linkerd using Gateway API, the more thoroughly convinced I am that the correct answer is that we should never ever touch any CRD where the API group isn't under linkerd.io.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions.

[brandonros]

/no-stale

[RafPe]

--set enableHttpRoutes=false

If I follow on that approach then the install refuses to continue ( using 2.17 )

❯ ./linkerd install | kubectl apply -f -
Linkerd CRDs must be installed first. Run linkerd install with the --crds flag.
error: no objects passed to apply

[fwernert]

same case

[wmorgan]

@RafPe @fwernert We've documented our recommendation on using the Gateway API. See https://linkerd.io/2.17/features/gateway-api/ .

If you are seeing the "Linkerd CRDs must be installed first" error, it means that linkerd install did not detect the Gateway API CRDs on your cluster. If you want another tool to manage these CRDs, not Linkerd, you will need to install them on the cluster prior to running that command.

[wmorgan]

In future releases we will remove this check so that you can complete the installation without the CRDs being present. See #13603