[Helm Chart] Change Kong Pod annotations to work out-of-the-box with Kuma and Istio

[fuljo]

What would you like to be added?

Since version 7.x the Kubernetes Dashboard Helm Chart includes a kong proxy to expose all the components from a single Service.
The following quote from the README suggests that this is intended to be an internal proxy, and one shall put an ingress controller or gateway to expose it.

We now use a single-container, DBless Kong installation as a gateway that connects all our containers and exposes the UI. Users can then use any ingress controller or proxy in front of kong gateway.

I suggest adding the following to the default helm chart values, so that the Kong Proxy gets incoming connections when inserted into Istio and Kuma meshes.

kong:
  # Enable inbound mesh proxying for Kuma and Istio
  podAnnotations:
    kuma.io/gateway: disabled
    traffic.sidecar.istio.io/includeInboundPorts: "*"

This snippet comes direcly from the README for the Kong Helm Chart, which specifies that:

Note: The default podAnnotations values disable inbound proxying for Kuma and Istio. This is appropriate when using Kong as a gateway for external traffic inbound into the cluster.

Alternative
If chaning the default behavior seems too much, we can:

• Add the above snippet to the values.yaml, but keep it commented.
  In this way users having problems with meshes can find an easier way to solve them.
• Add an entry in the Chart's documentation that explains this.

Why is this needed?

Since the README itself declares that the kong proxy is intended to be an internal proxy, we shall shipt it configured as such.
As explained above, the current behavior relies on the default values of the Kong chart, which configure it as an external gateway.

Current behavior in Istio
In Istio, one may configure this flow:

Gateway -- [HTTPRoute] --> Kong Proxy (dashboard)

When a request enters the Gateway, this is what happens:

1. the request is processed by the Gateway and sent outbound to the Kong Service
2. the isto-proxy on the Gateway intercepts the outbound request, "wraps" it with mTLS and sends it to the Kong Pod
3. The istio-proxy in the Kong Pod shall receive the mTLS message, "unwrap" it and send it to Kong in HTTP(S).
   However, the annotation traffic.sidecar.istio.io/includeInboundPorts: "" added to the Pod by the Kong Helm chart forces Istio to not set up traffic interception on any port.
   Therefore, the mTLS request is not "unwrapped" by Istio and goes through to Kong, which refuses it.

This is not immediate to debug, especially for someone that is not familiar with Istio's internals.