Allow weak key for openssl v3 (tls-enabled kafka)

[juvenn]

Since upgrading to vector 0.32 with bundled openssl from v1.1 to v3.0, it starts to refuse weak encryption algorithms for safety reasons (that's good). Specifically we are trying to sink messages to a tls-enabled kafka, which is unfortunately self-signed with weak key, thus, vector fails to verify certification during ssl handshake.

Upgrading sever side cert is not an option at the moment, so we are trying to figure out a way to allow weak key in vector. So could there be workaround to allow weak key for ssl to work? (Yes, we understand compromised security risks.) Probably by tweaking openssl config outlined here at librdkafka? Which I did try but it seems not work yet.

Setup

My setup is using docker image (timberio/vector:0.32.0-debian) with following vector config:

[sources.bind]
type = 'stdin'

[transforms.bind_decode]
  type = "remap"
  inputs = ["bind"]
  source = '''
  . = parse_json!(.message)
  '''

[sinks.bind_next_kafka]
  type = "kafka"
  inputs = [ "bind_decode" ]
  bootstrap_servers = "my-broker:9093"
  topic = "user-event"
  sasl.enabled = true
  sasl.mechanism = "SCRAM-SHA-256"
  sasl.username = "redacted"
  sasl.password = "redacted"
  encoding.codec = "json"
  tls.enabled = true
  tls.ca_file = "/etc/vector/my.pem"

vector fails to do ssl handshake during healthcheck.

2023-09-01T11:37:46.556755Z ERROR rdkafka::client: librdkafka: Global error: SSL (Local: SSL error): sasl_ssl://my-broker:9093/bootstrap: SSL handshake failed: ssl/statem/statem_clnt.c:1890: error:0A000086:SSL routines::certificate verify failed (after 26ms in state SSL_HANDSHAKE)
2023-09-01T11:37:46.556854Z ERROR rdkafka::client: librdkafka: Global error: AllBrokersDown (Local: All broker connections are down): 1/1 brokers are down
2023-09-01T11:37:47.538890Z ERROR rdkafka::client: librdkafka: Global error: SSL (Local: SSL error): sasl_ssl://my-broker:9093/bootstrap: SSL handshake failed: ssl/statem/statem_clnt.c:1890: error:0A000086:SSL routines::certificate verify failed (after
18ms in state SSL_HANDSHAKE, 1 identical error(s) suppressed)

Debug

With the vector container, there is openssl v3 bundled in. Using it to test revealed that CA certificate key too weak

openssl s_client -connect  my-broker:9093  -CAfile /etc/vector/my.pem

Peer signing digest: SHA256
Peer signature type: RSA
Server Temp Key: ECDH, prime256v1, 256 bits

---
SSL handshake has read 2054 bytes and written 484 bytes
Verification error: CA certificate key too weak
---

New, TLSv1.2, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES256-GCM-SHA384

Options tried

I tried to follow confluentinc/librdkafka#4204, by appending config to openssl conf

vim /usr/lib/ssl/openssl.conf
[default_conf]
ssl_conf = ssl_sect

[ssl_sect]
system_default = system_default_sect

[system_default_sect]
MinProtocol = TLSv1.2
CipherString = DEFAULT@SECLEVEL=1

run openssl s_connect -client to verify, it still reports key too weak error. I suspect this config snippet may be not correct.

[juvenn]

Okay, I made openssl to work by using this config (have to downgrade sec level to zero):

[openssl_init]
ssl_conf = ssl_sect

[ssl_sect]
system_default = system_default_sect

[system_default_sect]
MinProtocol = TLSv1.2
CipherString = DEFAULT@SECLEVEL=0

openssl now verifies ok,

SSL handshake has read 2054 bytes and written 486 bytes
Verification: OK

but vector still reports error,

2023-09-02T03:06:30.670833Z ERROR rdkafka::client: librdkafka: Global error: SSL (Local: SSL error): sasl_ssl://my-broker:9093/bootstrap: SSL handshake failed: ssl/statem/statem_clnt.c:1890: error:0A000086:SSL routines::certificate verify failed (after 18ms in state SSL_HANDSHAKE)

By explicitly specify openssl conf, vector works now:

export OPENSSL_CONF=/usr/lib/ssl/openssl.cnf
vector -v

[jszwedko]

Thanks for following up on this @juvenn . I think we are going to start pushing users to use openssl.cnf more rather than duplicate the options in Vector given there are a lot of openssl options. #18261 (comment) shows another example of this.