TLS gateway handshake error

[penghuazhou]

cluter1 and cluster2 should connect throw gateway, but is throw this error.

[1] 2024/09/30 02:53:46.049283 [INF] 10.2.96.4:11013 - gid:233 - Processing inbound gateway connection
[1] 2024/09/30 02:53:46.049297 [DBG] 10.2.96.4:11013 - gid:233 - Starting TLS gateway server handshake
[1] 2024/09/30 02:53:46.049355 [ERR] 10.2.96.4:11013 - gid:233 - TLS gateway handshake error: read tcp 172.17.2.131:7222->10.2.96.4:11013: read: connection reset by peer
[1] 2024/09/30 02:53:46.049364 [INF] 10.2.96.4:11013 - gid:233 - Gateway connection closed: TLS Handshake Failure

cluter1 config:

apiVersion: v1
data:
  nats.conf: |
    {
      "cluster": {
        "name": "cluster1",
        "no_advertise": true,
        "port": 6222,
        "routes": [
          "nats://nats-0.nats-headless.infra-al-cn-gdsz-5.svc.cluster.local:6222"
        ]
      },
      "gateway": {
        "gateways": [
          {
            "name": "cluster2",
            "urls": [
              "nats://10.21.147.219:7222"
            ]
          }
        ],
        "name": "cluster1",
        "port": 7222,
        "tls": {
          "ca_file": "/etc/nats-ca-cert/ca.crt",
          "cert_file": "/etc/nats-certs/gateway/client.crt",
          "key_file": "/etc/nats-certs/gateway/client.key",
          "verify_and_map": true
        },
        "authorization": {
          "user" = "O=mkcert development certificate, OU=MacBook-Pro-72.local"
        }
      },
      "http_port": 8222,
      "lame_duck_duration": "30s",
      "lame_duck_grace_period": "10s",
      "pid_file": "/var/run/nats/nats.pid",
      "port": 4222,
      "server_name": $SERVER_NAME
    }
kind: ConfigMap
metadata:
  name: nats-config
  namespace: infra-al-cn-gdsz-5

use below command to generate cluster1 cert.
mkcert -client
-key-file key.pem
-cert-file cert.pem
10.81.159.4

cluter2 config:

apiVersion: v1
data:
  nats.conf: |
    {
      "cluster": {
        "name": "cluster2",
        "no_advertise": true,
        "port": 6222,
        "routes": [
          "nats://nats-0.nats-headless.infra-al-us-iad-2.svc.cluster.local:6222"
        ]
      },
      "gateway": {
        "gateways": [
          {
            "name": "cluster1",
            "urls": [
              "nats://120.24.147.219:7222"
            ]
          }
        ],
        "name": "cluster2",
        "port": 7222,
        "tls": {
          "ca_file": "/etc/nats-ca-cert/ca.crt",
          "cert_file": "/etc/nats-certs/gateway/client.crt",
          "key_file": "/etc/nats-certs/gateway/client.key",
          "verify_and_map": true
        },
        "authorization": {
          "user" = "O=mkcert development certificate, OU=MacBook-Pro-72.local"
        }
      },
      "http_port": 8222,
      "lame_duck_duration": "30s",
      "lame_duck_grace_period": "10s",
      "pid_file": "/var/run/nats/nats.pid",
      "port": 4222,
      "server_name": $SERVER_NAME
    }
kind: ConfigMap
metadata:
  name: nats-config
  namespace: infra-al-us-iad-2

use below command to generate cluster2 cert.
mkcert -client
-key-file key.pem
-cert-file cert.pem
10.21.147.219

[derekcollison]

This is just understanding how certs and CAs work. @wallyqs may be able to help.

[wallyqs]

There are a few things that are problematic in the config, the gateway are discovering internal IPs so trying to connect to each other meaning that you need to set advertise in order to be able to gossip the proper IP. Also noticed that you are setting nats-0 in order to discover the cluster which will work but will rely on nats-0 being up (nats://nats-0.nats-headless.infra-al-cn-gdsz-5.svc.cluster.local:6222).