rabbitmq does not work in IPv6 environments #1225
Description
I want to verify that rabbimq works properly on ipv6 network protocol stack.
I'm k8s v1.23.8 deployed rabbimq cluster operator (https://github.com/rabbitmq/cluster-operator/releases/tag/v2.1.0),
Then deploy rabbitmq:3.10.2-management, which is the default for operator.
By default rabbitmq works properly
[middleware-develop-6852]$ kubectl -n rabbitmq-system get pods
NAME READY STATUS RESTARTS AGE
my-rabbitmq-server-0 1/1 Running 0 37m
my-rabbitmq-server-1 1/1 Running 0 37m
my-rabbitmq-server-2 1/1 Running 0 37m
I checked the rabbitmq configuration and the DNS related configuration as follows
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- ls -al /etc/rabbitmq
total 4
drwxrwxrwx 1 rabbitmq rabbitmq 20 May 20 2022 .
drwxr-xr-x 1 root root 38 Jan 7 12:13 ..
drwxrwxrwx 1 rabbitmq rabbitmq 106 Jan 7 12:13 conf.d
-rw-r--r-- 1 root root 43 May 20 2022 enabled_plugins
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- ls -al /etc/rabbitmq/conf.d
total 16
drwxrwxrwx 1 rabbitmq rabbitmq 106 Jan 7 12:13 .
drwxrwxrwx 1 rabbitmq rabbitmq 20 May 20 2022 ..
-rw-rw-r-- 1 rabbitmq rabbitmq 491 May 20 2022 10-defaults.conf
-rw-r--r-- 1 root root 459 Jan 7 12:12 10-operatorDefaults.conf
-rw-r--r-- 1 root root 96 Jan 7 12:12 11-default_user.conf
-rw-r--r-- 1 root root 51 Jan 7 12:12 90-userDefinedConfiguration.conf
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- cat /etc/rabbitmq/enabled_plugins
[rabbitmq_management,rabbitmq_prometheus].
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- cat /etc/rabbitmq/conf.d/10-defaults.conf
## DEFAULT SETTINGS ARE NOT MEANT TO BE TAKEN STRAIGHT INTO PRODUCTION
## see https://www.rabbitmq.com/configure.html for further information
## on configuring RabbitMQ
## allow access to the guest user from anywhere on the network
## https://www.rabbitmq.com/access-control.html#loopback-users
## https://www.rabbitmq.com/production-checklist.html#users
loopback_users.guest = false
## Send all logs to stdout/TTY. Necessary to see logs when running via
## a container
log.console = true
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- cat /etc/rabbitmq/conf.d/10-operatorDefaults.conf
queue_master_locator = min-masters
disk_free_limit.absolute = 2GB
cluster_partition_handling = pause_minority
cluster_formation.peer_discovery_backend = rabbit_peer_discovery_k8s
cluster_formation.k8s.host = kubernetes.default
cluster_formation.k8s.address_type = hostname
cluster_formation.target_cluster_size_hint = 3
cluster_name = my-rabbitmq
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- cat /etc/rabbitmq/conf.d/11-default_user.conf
default_user = default_user_HHzAT0LOPmQrBZDDbtK
default_pass = rtfO1IwUgiaNHG-LPe0r1inbNlZ9WJu9
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- cat /etc/rabbitmq/conf.d/90-userDefinedConfiguration.conf
total_memory_available_override_value = 1717986919
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- cat /etc/hosts
# Kubernetes-managed hosts file.
127.0.0.1 localhost
::1 localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
fe00::0 ip6-mcastprefix
fe00::1 ip6-allnodes
fe00::2 ip6-allrouters
10.3.203.129 my-rabbitmq-server-0.my-rabbitmq-nodes.rabbitmq-system.svc.cluster.local my-rabbitmq-server-0
fd00:10:16::cb81 my-rabbitmq-server-0.my-rabbitmq-nodes.rabbitmq-system.svc.cluster.local my-rabbitmq-server-0
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- cat /etc/resolv.conf
nameserver 10.4.0.10
search rabbitmq-system.svc.cluster.local svc.cluster.local cluster.local
options ndots:5
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- hostname
my-rabbitmq-server-0
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- nslookup my-rabbitmq-server-0.my-rabbitmq-nodes.rabbitmq-system.svc.cluster.local
Server: 10.4.0.10
Address: 10.4.0.10#53
Name: my-rabbitmq-server-0.my-rabbitmq-nodes.rabbitmq-system.svc.cluster.local
Address: 10.3.203.129
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- nslookup my-rabbitmq-server-0.my-rabbitmq-nodes
Server: 10.4.0.10
Address: 10.4.0.10#53
Name: my-rabbitmq-server-0.my-rabbitmq-nodes.rabbitmq-system.svc.cluster.local
Address: 10.3.203.129
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- nslookup my-rabbitmq-server-0
Server: 10.4.0.10
Address: 10.4.0.10#53
** server can't find my-rabbitmq-server-0: NXDOMAIN
command terminated with exit code 1
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- rabbitmqctl eval 'inet:gethostbyname("my-rabbitmq-server-0.my-rabbitmq-nodes.rabbitmq-system.svc.cluster.local", inet).'
{ok,{hostent,"my-rabbitmq-server-0.my-rabbitmq-nodes.rabbitmq-system.svc.cluster.local",
["my-rabbitmq-server-0"],
inet,4,
[{10,3,203,129}]}}
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- rabbitmqctl eval 'inet:gethostbyname("my-rabbitmq-server-0.my-rabbitmq-nodes", inet).'
{ok,{hostent,"my-rabbitmq-server-0.my-rabbitmq-nodes.rabbitmq-system.svc.cluster.local",
[],inet,4,
[{10,3,203,129}]}}
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- rabbitmqctl eval 'inet:gethostbyname("my-rabbitmq-server-0", inet).'
{ok,{hostent,"my-rabbitmq-server-0.my-rabbitmq-nodes.rabbitmq-system.svc.cluster.local",
["my-rabbitmq-server-0"],
inet,4,
[{10,3,203,129}]}}
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- rabbitmqctl eval 'inet:gethostbyname("my-rabbitmq-server-0.my-rabbitmq-nodes.rabbitmq-system.svc.cluster.local", inet6).'
{ok,{hostent,"my-rabbitmq-server-0.my-rabbitmq-nodes.rabbitmq-system.svc.cluster.local",
[],inet6,16,
[{64768,16,22,0,0,0,0,52097}]}}
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- rabbitmqctl eval 'inet:gethostbyname("my-rabbitmq-server-0.my-rabbitmq-nodes", inet6).'
{error,nxdomain}
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- rabbitmqctl eval 'inet:gethostbyname("my-rabbitmq-server-0", inet6).'
{ok,{hostent,"my-rabbitmq-server-0.my-rabbitmq-nodes.rabbitmq-system.svc.cluster.local",
[],inet6,16,
[{64768,16,22,0,0,0,0,52097}]}}
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- rabbitmqctl eval 'inet_res:resolve("my-rabbitmq-server-0.my-rabbitmq-nodes.rabbitmq-system.svc.cluster.local", in, a).'
{ok,{dns_rec,{dns_header,2,true,query,true,false,true,false,false,0},
[{dns_query,"my-rabbitmq-server-0.my-rabbitmq-nodes.rabbitmq-system.svc.cluster.local",
a,in,false}],
[{dns_rr,"my-rabbitmq-server-0.my-rabbitmq-nodes.rabbitmq-system.svc.cluster.local",
a,in,0,21,
{10,3,203,129},
undefined,[],false}],
[],[]}}
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- rabbitmqctl eval 'inet_res:resolve("my-rabbitmq-server-0.my-rabbitmq-nodes", in, a).'
{error,nxdomain}
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- rabbitmqctl eval 'inet_res:resolve("my-rabbitmq-server-0", in, a).'
{error,nxdomain}
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- rabbitmqctl eval 'inet_res:resolve("my-rabbitmq-server-0.my-rabbitmq-nodes.rabbitmq-system.svc.cluster.local", in, aaaa).'
{ok,{dns_rec,{dns_header,5,true,query,true,false,true,false,false,0},
[{dns_query,"my-rabbitmq-server-0.my-rabbitmq-nodes.rabbitmq-system.svc.cluster.local",
aaaa,in,false}],
[],
[{dns_rr,"cluster.local",soa,in,0,30,
{"ns.dns.cluster.local","hostmaster.cluster.local",
1673094992,7200,1800,86400,30},
undefined,[],false}],
[]}}
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- rabbitmqctl eval 'inet_res:resolve("my-rabbitmq-server-0.my-rabbitmq-nodes", in, aaaa).'
{error,nxdomain}
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- rabbitmqctl eval 'inet_res:resolve("my-rabbitmq-server-0", in, aaaa).'
{error,nxdomain}
The second verification is that the /etc/rabbitmq-erl_inetrc file is added to the image,
but is not configured to enable ipv6
[middleware-develop-6852]$ cat /etc/rabbitmq/erl_inetrc
{inet6, true}.
{resolv_conf, "/etc/resolv.conf"}.
{lookup, [dns]}.
rabbitmq will also work, and the rest of the configuration is the same as before
[middleware-develop-6852]$ kubectl -n rabbitmq-system get pods
NAME READY STATUS RESTARTS AGE
my-rabbitmq-server-0 1/1 Running 0 104s
my-rabbitmq-server-1 1/1 Running 0 104s
my-rabbitmq-server-2 1/1 Running 0 104s
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- cat /etc/hosts
# Kubernetes-managed hosts file.
127.0.0.1 localhost
::1 localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
fe00::0 ip6-mcastprefix
fe00::1 ip6-allnodes
fe00::2 ip6-allrouters
10.3.203.202 my-rabbitmq-server-0.my-rabbitmq-nodes.rabbitmq-system.svc.cluster.local my-rabbitmq-server-0
fd00:10:16::cbca my-rabbitmq-server-0.my-rabbitmq-nodes.rabbitmq-system.svc.cluster.local my-rabbitmq-server-0
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- cat /etc/resolv.conf
nameserver 10.4.0.10
search rabbitmq-system.svc.cluster.local svc.cluster.local cluster.local
options ndots:5
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- hostname
my-rabbitmq-server-0
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- nslookup my-rabbitmq-server-0.my-rabbitmq-nodes.rabbitmq-system.svc.cluster.local
Server: 10.4.0.10
Address: 10.4.0.10#53
Name: my-rabbitmq-server-0.my-rabbitmq-nodes.rabbitmq-system.svc.cluster.local
Address: 10.3.203.202
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- nslookup my-rabbitmq-server-0.my-rabbitmq-nodes
Server: 10.4.0.10
Address: 10.4.0.10#53
Name: my-rabbitmq-server-0.my-rabbitmq-nodes.rabbitmq-system.svc.cluster.local
Address: 10.3.203.202
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- nslookup my-rabbitmq-server-0
Server: 10.4.0.10
Address: 10.4.0.10#53
** server can't find my-rabbitmq-server-0: NXDOMAIN
command terminated with exit code 1
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- rabbitmqctl eval 'inet:gethostbyname("my-rabbitmq-server-0.my-rabbitmq-nodes.rabbitmq-system.svc.cluster.local", inet).'
{ok,{hostent,"my-rabbitmq-server-0.my-rabbitmq-nodes.rabbitmq-system.svc.cluster.local",
["my-rabbitmq-server-0"],
inet,4,
[{10,3,203,202}]}}
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- rabbitmqctl eval 'inet:gethostbyname("my-rabbitmq-server-0.my-rabbitmq-nodes", inet).'
{ok,{hostent,"my-rabbitmq-server-0.my-rabbitmq-nodes.rabbitmq-system.svc.cluster.local",
[],inet,4,
[{10,3,203,202}]}}
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- rabbitmqctl eval 'inet:gethostbyname("my-rabbitmq-server-0", inet).'
{ok,{hostent,"my-rabbitmq-server-0.my-rabbitmq-nodes.rabbitmq-system.svc.cluster.local",
["my-rabbitmq-server-0"],
inet,4,
[{10,3,203,202}]}}
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- rabbitmqctl eval 'inet:gethostbyname("my-rabbitmq-server-0.my-rabbitmq-nodes.rabbitmq-system.svc.cluster.local", inet6).'
{ok,{hostent,"my-rabbitmq-server-0.my-rabbitmq-nodes.rabbitmq-system.svc.cluster.local",
[],inet6,16,
[{64768,16,22,0,0,0,0,52170}]}}
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- rabbitmqctl eval 'inet:gethostbyname("my-rabbitmq-server-0.my-rabbitmq-nodes", inet6).'
{error,nxdomain}
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- rabbitmqctl eval 'inet:gethostbyname("my-rabbitmq-server-0", inet6).'
{ok,{hostent,"my-rabbitmq-server-0.my-rabbitmq-nodes.rabbitmq-system.svc.cluster.local",
[],inet6,16,
[{64768,16,22,0,0,0,0,52170}]}}
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- rabbitmqctl eval 'inet_res:resolve("my-rabbitmq-server-0.my-rabbitmq-nodes.rabbitmq-system.svc.cluster.local", in, a).'
{ok,{dns_rec,{dns_header,1,true,query,true,false,true,false,false,0},
[{dns_query,"my-rabbitmq-server-0.my-rabbitmq-nodes.rabbitmq-system.svc.cluster.local",
a,in,false}],
[{dns_rr,"my-rabbitmq-server-0.my-rabbitmq-nodes.rabbitmq-system.svc.cluster.local",
a,in,0,30,
{10,3,203,202},
undefined,[],false}],
[],[]}}
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- rabbitmqctl eval 'inet_res:resolve("my-rabbitmq-server-0.my-rabbitmq-nodes", in, a).'
{error,nxdomain}
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- rabbitmqctl eval 'inet_res:resolve("my-rabbitmq-server-0", in, a).'
{error,nxdomain}
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- rabbitmqctl eval 'inet_res:resolve("my-rabbitmq-server-0.my-rabbitmq-nodes.rabbitmq-system.svc.cluster.local", in, aaaa).'
{ok,{dns_rec,{dns_header,4,true,query,true,false,true,false,false,0},
[{dns_query,"my-rabbitmq-server-0.my-rabbitmq-nodes.rabbitmq-system.svc.cluster.local",
aaaa,in,false}],
[],
[{dns_rr,"cluster.local",soa,in,0,30,
{"ns.dns.cluster.local","hostmaster.cluster.local",
1673097281,7200,1800,86400,30},
undefined,[],false}],
[]}}
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- rabbitmqctl eval 'inet_res:resolve("my-rabbitmq-server-0.my-rabbitmq-nodes", in, aaaa).'
{error,nxdomain}
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- rabbitmqctl eval 'inet_res:resolve("my-rabbitmq-server-0", in, aaaa).'
{error,nxdomain}
The third verification is that you are adding the /etc/rabbitmq-erl_inetrc file to the image and enabling ipv6
In this case, rabbitmq does not work properly. The rabbitmq logs are as follows：
2023-01-07 13:30:08.123689+00:00 [info] <0.219.0> Configured peer discovery backend: rabbit_peer_discovery_k8s
2023-01-07 13:30:08.124245+00:00 [info] <0.219.0> Will try to lock with peer discovery backend rabbit_peer_discovery_k8s
2023-01-07 13:30:08.127846+00:00 [warning] <0.315.0> Description: "Authenticity is not established by certificate path validation"
2023-01-07 13:30:08.127846+00:00 [warning] <0.315.0> Reason: "Option {verify, verify_peer} and cacertfile/cacerts is missing"
2023-01-07 13:30:08.127846+00:00 [warning] <0.315.0>
2023-01-07 13:30:08.185568+00:00 [error] <0.219.0> Failed to fetch a list of nodes from Kubernetes API: {failed_connect,[{to_address,{"kubernetes.default",443}},
2023-01-07 13:30:08.185568+00:00 [error] <0.219.0> {inet6,[inet6],nxdomain}]}
2023-01-07 13:30:08.188142+00:00 [warning] <0.316.0> Description: "Authenticity is not established by certificate path validation"
2023-01-07 13:30:08.188142+00:00 [warning] <0.316.0> Reason: "Option {verify, verify_peer} and cacertfile/cacerts is missing"
2023-01-07 13:30:08.188142+00:00 [warning] <0.316.0>
2023-01-07 13:30:08.219100+00:00 [error] <0.219.0> Failed to lock with peer discovery backend rabbit_peer_discovery_k8s: "{failed_connect,[{to_address,{\"kubernetes.default\",443}},\n {inet6,[inet6],nxdomain}]}"
2023-01-07 13:30:38.220353+00:00 [info] <0.219.0> Will try to lock with peer discovery backend rabbit_peer_discovery_k8s
2023-01-07 13:30:38.223474+00:00 [warning] <0.365.0> Description: "Authenticity is not established by certificate path validation"
2023-01-07 13:30:38.223474+00:00 [warning] <0.365.0> Reason: "Option {verify, verify_peer} and cacertfile/cacerts is missing"
2023-01-07 13:30:38.223474+00:00 [warning] <0.365.0>
2023-01-07 13:30:38.259894+00:00 [error] <0.219.0> Failed to fetch a list of nodes from Kubernetes API: {failed_connect,[{to_address,{"kubernetes.default",443}},
2023-01-07 13:30:38.259894+00:00 [error] <0.219.0> {inet6,[inet6],nxdomain}]}
2023-01-07 13:30:38.262612+00:00 [warning] <0.366.0> Description: "Authenticity is not established by certificate path validation"
2023-01-07 13:30:38.262612+00:00 [warning] <0.366.0> Reason: "Option {verify, verify_peer} and cacertfile/cacerts is missing"
2023-01-07 13:30:38.262612+00:00 [warning] <0.366.0>
2023-01-07 13:30:38.301716+00:00 [error] <0.219.0> Failed to lock with peer discovery backend rabbit_peer_discovery_k8s: "{failed_connect,[{to_address,{\"kubernetes.default\",443}},\n {inet6,[inet6],nxdomain}]}"
2023-01-07 13:31:08.302540+00:00 [info] <0.219.0> Will try to lock with peer discovery backend rabbit_peer_discovery_k8s
2023-01-07 13:31:08.308443+00:00 [warning] <0.415.0> Description: "Authenticity is not established by certificate path validation"
2023-01-07 13:31:08.308443+00:00 [warning] <0.415.0> Reason: "Option {verify, verify_peer} and cacertfile/cacerts is missing"
2023-01-07 13:31:08.308443+00:00 [warning] <0.415.0>
2023-01-07 13:31:08.345506+00:00 [error] <0.219.0> Failed to fetch a list of nodes from Kubernetes API: {failed_connect,[{to_address,{"kubernetes.default",443}},
2023-01-07 13:31:08.345506+00:00 [error] <0.219.0> {inet6,[inet6],nxdomain}]}
2023-01-07 13:31:08.350786+00:00 [warning] <0.416.0> Description: "Authenticity is not established by certificate path validation"
2023-01-07 13:31:08.350786+00:00 [warning] <0.416.0> Reason: "Option {verify, verify_peer} and cacertfile/cacerts is missing"
2023-01-07 13:31:08.350786+00:00 [warning] <0.416.0>
2023-01-07 13:31:08.387389+00:00 [error] <0.219.0> Failed to lock with peer discovery backend rabbit_peer_discovery_k8s: "{failed_connect,[{to_address,{\"kubernetes.default\",443}},\n {inet6,[inet6],nxdomain}]}"
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- ls -al /etc/rabbitmq
total 12
drwxrwxrwx 1 rabbitmq rabbitmq 45 Jan 7 13:29 .
drwxr-xr-x 1 root root 38 Jan 7 13:29 ..
drwxrwxrwx 1 rabbitmq rabbitmq 106 Jan 7 13:29 conf.d
-rw-r--r-- 1 root root 43 May 20 2022 enabled_plugins
-rw-r--r-- 1 rabbitmq rabbitmq 67 Jan 7 11:05 erl_inetrc
-rw-r--r-- 1 root root 148 Jan 7 13:29 rabbitmq-env.conf
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- cat /etc/rabbitmq/erl_inetrc
{inet6, true}.
{resolv_conf, "/etc/resolv.conf"}.
{lookup, [dns]}.
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- cat /etc/rabbitmq/rabbitmq-env.conf
RABBITMQ_SERVER_ADDITIONAL_ERL_ARGS="-kernel inetrc '/etc/rabbitmq/erl_inetrc' -proto_dist inet6_tcp"
RABBITMQ_CTL_ERL_ARGS="-proto_dist inet6_tcp"
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- cat /etc/hosts
# Kubernetes-managed hosts file.
127.0.0.1 localhost
::1 localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
fe00::0 ip6-mcastprefix
fe00::1 ip6-allnodes
fe00::2 ip6-allrouters
10.3.203.238 my-rabbitmq-server-0.my-rabbitmq-nodes.rabbitmq-system.svc.cluster.local my-rabbitmq-server-0
fd00:10:16::cbee my-rabbitmq-server-0.my-rabbitmq-nodes.rabbitmq-system.svc.cluster.local my-rabbitmq-server-0
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- cat /etc/resolv.conf
nameserver 10.4.0.10
search rabbitmq-system.svc.cluster.local svc.cluster.local cluster.local
options ndots:5
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- hostname
my-rabbitmq-server-0
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- nslookup my-rabbitmq-server-0.my-rabbitmq-nodes.rabbitmq-system.svc.cluster.local
Server: 10.4.0.10
Address: 10.4.0.10#53
Name: my-rabbitmq-server-0.my-rabbitmq-nodes.rabbitmq-system.svc.cluster.local
Address: fd00:10:16::cbee
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- nslookup my-rabbitmq-server-0.my-rabbitmq-nodes
Server: 10.4.0.10
Address: 10.4.0.10#53
Name: my-rabbitmq-server-0.my-rabbitmq-nodes.rabbitmq-system.svc.cluster.local
Address: fd00:10:16::cbee
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- nslookup my-rabbitmq-server-0
Server: 10.4.0.10
Address: 10.4.0.10#53
** server can't find my-rabbitmq-server-0: NXDOMAIN
command terminated with exit code 1
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- rabbitmqctl eval 'inet:gethostbyname("my-rabbitmq-server-0.my-rabbitmq-nodes.rabbitmq-system.svc.cluster.local", inet).'
{ok,{hostent,"my-rabbitmq-server-0.my-rabbitmq-nodes.rabbitmq-system.svc.cluster.local",
[],inet,4,
[{127,0,0,1}]}}
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- rabbitmqctl eval 'inet:gethostbyname("my-rabbitmq-server-0.my-rabbitmq-nodes", inet).'
{error,nxdomain}
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- rabbitmqctl eval 'inet:gethostbyname("my-rabbitmq-server-0", inet).'
{ok,{hostent,"my-rabbitmq-server-0",[],inet,4,[{127,0,0,1}]}}
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- rabbitmqctl eval 'inet:gethostbyname("my-rabbitmq-server-0.my-rabbitmq-nodes.rabbitmq-system.svc.cluster.local", inet6).'
{ok,{hostent,"my-rabbitmq-server-0.my-rabbitmq-nodes.rabbitmq-system.svc.cluster.local",
[],inet6,16,
[{64768,16,22,0,0,0,0,52206}]}}
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- rabbitmqctl eval 'inet:gethostbyname("my-rabbitmq-server-0.my-rabbitmq-nodes", inet6).'
{ok,{hostent,"my-rabbitmq-server-0.my-rabbitmq-nodes.rabbitmq-system.svc.cluster.local",
[],inet6,16,
[{64768,16,22,0,0,0,0,52206}]}}
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- rabbitmqctl eval 'inet:gethostbyname("my-rabbitmq-server-0", inet6).'
{ok,{hostent,"my-rabbitmq-server-0",[],inet6,16,[{0,0,0,0,0,0,0,1}]}}
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- rabbitmqctl eval 'inet_res:resolve("my-rabbitmq-server-0.my-rabbitmq-nodes.rabbitmq-system.svc.cluster.local", in, a).'
{ok,{dns_rec,{dns_header,5,true,query,true,false,true,false,false,0},
[{dns_query,"my-rabbitmq-server-0.my-rabbitmq-nodes.rabbitmq-system.svc.cluster.local",
a,in,false}],
[],
[{dns_rr,"cluster.local",soa,in,0,30,
{"ns.dns.cluster.local","hostmaster.cluster.local",
1673098842,7200,1800,86400,30},
undefined,[],false}],
[]}}
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- rabbitmqctl eval 'inet_res:resolve("my-rabbitmq-server-0.my-rabbitmq-nodes", in, a).'
{error,nxdomain}
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- rabbitmqctl eval 'inet_res:resolve("my-rabbitmq-server-0", in, a).'
{error,nxdomain}
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- rabbitmqctl eval 'inet_res:resolve("my-rabbitmq-server-0.my-rabbitmq-nodes.rabbitmq-system.svc.cluster.local", in, aaaa).'
{ok,{dns_rec,{dns_header,24,true,query,true,false,true,false,false,0},
[{dns_query,"my-rabbitmq-server-0.my-rabbitmq-nodes.rabbitmq-system.svc.cluster.local",
aaaa,in,false}],
[{dns_rr,"my-rabbitmq-server-0.my-rabbitmq-nodes.rabbitmq-system.svc.cluster.local",
aaaa,in,0,30,
{64768,16,22,0,0,0,0,52206},
undefined,[],false}],
[],[]}}
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- rabbitmqctl eval 'inet_res:resolve("my-rabbitmq-server-0.my-rabbitmq-nodes", in, aaaa).'
{error,nxdomain}
[middleware-develop-6852]$ kubectl -n rabbitmq-system exec -ti my-rabbitmq-server-0 -- rabbitmqctl eval 'inet_res:resolve("my-rabbitmq-server-0", in, aaaa).'
{error,nxdomain}
What is the problem and how should I solve it? Or what verification do I need to do