Node fails to connect to the K8S API endpoint in a FIPS-only environment

[manoj365study]

Describe the bug

We have tried to install the Rabbitmq on OCP platform.

When the peer discovery mechanism used is rabbit_peer_discovery_k8s . The installation was successful. But when the Rabbitmq pods are keep on restarting with the error

*2022-11-14 07:42:49.501027+00:00 [error] <0.230.0> Failed to lock with peer discovery backend rabbit_peer_discovery_k8s: "{failed_connect,\n [{to_address,{"kubernetes.default.svc.cluster.local",443}},\n {inet,\n [inet],\n {eoptions,\n {{badarg,\n [

{crypto,evp_generate_key_nif,[x25519,undefined],[]}
,\n {ssl_cipher,generate_client_shares,2,\n [

{file,"ssl_cipher.erl"}
,{line,1265}]},\n {ssl_gen_statem,initial_hello,3,\n [

{file,"ssl_gen_statem.erl"}
,{line,457}]},\n {gen_statem,loop_state_callback,11,\n [

{file,"gen_statem.erl"}
,{line,1203}]},\n {tls_connection,init,1,\n [

{file,"tls_connection.erl"}
,{line,154}]},\n {proc_lib,init_p_do_apply,3,\n [

{file,"proc_lib.erl"}
,{line,226}]}]},\n {gen_statem,call,[<0.754.0>,

{start,2250}
,infinity]}}}}]}"*

And we don't see this issue when we provide the peer discovery mechanism as classic config. We need to know why we are seeing this error, if the peer discovery mechanism is not classic config. Also what is the recommended peer discovery mechanism that should be used for CRMQ ?

There are reported problems in RabbitMQ open source to work with a FIPS enabled K8s cluster, when using the peer discovery k8s cluster formation method.

The underlying problem is from Erlang, but there is no formal word about how to make this combination work.

Can someone tell when Erlang will gain FIPS mode compatibility?

Reproduction steps

Install Rabbitmq in FIPS enabled OCP cluster.

Expected behavior

Even when FIPS enabled OCP cluster Rabbitmq should install with peer discovery mechanism as rabbit_peer_discovery_k8s

Additional context

No response

[michaelklishin]

The node could not connect to the Kubernetes API endpoint at kubernetes.default.svc.cluster.local because of an exception in the crypto module. Seemingly ECC 25519 is involved.

When you use classic config peer discovery, nodes do not contact any external services.

Sounds like you need a FIPS-enabled Erlang runtime. Our team will soon provide a FIPS-enabled build with OpenSSL 3 but as part of the commercial edition only. Open source RabbitMQ and Erlang users must build (or, well, local an existing binary build) a compatible Erlang version with FIPS enabled.

[michaelklishin]

Can someone tell when Erlang will gain FIPS mode compatibility?

Erlang has gained FIPS compatibility a decade ago. Then it took a while for it to get rebuilt on top of OpenSSL 3. RabbitMQ's Erlang compatibility guide does mention what modern version supports FIPS with OpenSSL 3.

That does not mean that you can just use any Erlang 26.1 or 26.2 version. The FIPS mode must be enabled at compilation time.

[michaelklishin]

@manoj4rabbbitmq please stop repeatedly filing the same issue or you will be banned from ever filing a new issue in this org.