vector support for sasl.mechanism = oauthbearer with librdkafka

[CH-raghavendergogi]

Hi Vector Team,
Just wondering if there is support for sasl.mechanism = oauthbearer which comes from confluent kafka. Because I ran into following error while configuring sasl properties under librdkafka_options: inside vector.yaml.
2024-07-27T22:13:31.673098Z ERROR vector::topology::builder: Configuration error. error=Sink "logging_kafka": creating kafka producer failed: Client config error: Configuration property "sasl.oauthbearer.client.id" not supported in this build: OAuth/OIDC depends on libcurl and OpenSSL which were not available at build time sasl.oauthbearer.client.id 0oafq15axz9WMbkJd1d7

[jszwedko]

Can you provide the output of vector --version?

[CH-raghavendergogi]

testvaultnodetesting core # /opt/sbin/vector --version
vector 0.34.0 (x86_64-unknown-linux-gnu c909b66 2023-11-07 15:07:26.748571656)

[jszwedko]

Thanks! I was wondering if you might be using the musl-based binary which has less rdkafka features, but you aren't. I think we may need to include an additional feature flag to enable use of sasl.oauthbearer.client.id: https://github.com/fede1024/rust-rdkafka/blob/master/Cargo.toml#L62-L63

[jszwedko]

Testing out this change over here: #20970

[CH-raghavendergogi]

[jszwedko]

I created a one-off build to test out linking in curl. Would you want to give that a shot? You can find the binaries here: https://packages.timber.io/vector/custom/0.41.0.custom.b486506/

I also found fede1024/rust-rdkafka#694 which seems to be related.

[CH-raghavendergogi]

@jszwedko here is the update from our team with new build. Sorry for delayed response.
There's an open issue with librdkafka and it not picking up CURL options, confluentinc/librdkafka#4761
#4761 Failed to acquire SASL OAUTHBEARER token: SSL certificate problem: unable to get local issuer certificate
Description
I am trying to set sasl.oauthbearer.token.endpoint.url to https endpoint
if (rd_kafka_conf_set(conf, "sasl.oauthbearer.token.endpoint.url", "https://IP:https_port/realms/master/protocol/openid-connect/token", errstr, sizeof(errstr)) != RD_KAFKA_CONF_OK) {
fprintf(stderr, "Failed to set sasl.oauthbearer.token.endpoint.url: %s\n", errstr);
return 1;
}

the above code fails while trying to connect the oauth token provider with error
%3|1718691008.556|OIDC|rdkafka#producer-1| [thrd:background]: Failed to retrieve OIDC token from "https://IP:https_port/realms/master/protocol/openid-connect/token": SSL certificate problem: unable to get local issuer certificate (-1)
%3|1718691008.561|ERROR|rdkafka#producer-1| [thrd:background]: Failed to acquire SASL OAUTHBEARER token: SSL certificate problem: unable to get local issuer certificate

When using the same system, the curl command functions correctly when I configure CURL_CA_BUNDLE to the CA certificate file from the OAuth token provider. I attempted to execute the librdkafka sample with the CURL_CA_BUNDLE environment variable, but unfortunately, it did not work. Consequently, I had to make corrections to the librdkafka code, specifically in rdhttp.c, where I needed to
rd_http_error_t *rd_http_req_init(rd_http_req_t *hreq, const char *url) {
const char *ca_bundle_path = getenv("CURL_CA_BUNDLE");
if (ca_bundle_path) {
curl_easy_setopt(hreq->hreq_curl, CURLOPT_CAINFO, ca_bundle_path);
}
...
}

since the librdkafka uses the curl I was thinking the curl environment variables should work.

How to reproduce
set sasl.oauthbearer.token.endpoint.url to a https endpoint
Checklist
• librdkafka version (release number or git tag): v2.4.0
• Apache Kafka version: ``
• librdkafka client configuration: sasl.oauthbearer.token.endpoint.url=https://url/
• Operating system: ubuntu
• Provide logs (with debug=.. as necessary) from librdkafka
%7|1718710484.974|SASL|rdkafka#producer-1| [thrd:app]: Selected provider OAUTHBEARER (builtin) for SASL mechanism OAUTHBEARER
%7|1718710484.974|OPENSSL|rdkafka#producer-1| [thrd:app]: Using OpenSSL version OpenSSL 3.0.2 15 Mar 2022 (0x30000020, librdkafka built with 0x30000020)
%7|1718710484.977|SSL|rdkafka#producer-1| [thrd:app]: Loading CA certificate(s) from file ./ca.crt
%7|1718710484.979|INIT|rdkafka#producer-1| [thrd:app]: librdkafka v2.4.0-RC2-dirty (0x20400ff) rdkafka#producer-1 initialized (builtin.features gzip,snappy,ssl,sasl,regex,lz4,sasl_gssapi,sasl_plain,sasl_scram,plugins,sasl_oauthbearer,http,oidc, GCC GXX PKGCONFIG INSTALL GNULD LDS C11THREADS LIBDL PLUGINS ZLIB SSL SASL_CYRUS CURL HDRHISTOGRAM SYSLOG SNAPPY SOCKEM SASL_SCRAM SASL_OAUTHBEARER OAUTHBEARER_OIDC CRC32C_HW, debug 0x200)
% Type some text and hit enter to produce message
% Or just hit enter to only serve delivery reports
% Press Ctrl-C or Ctrl-D to exit
%3|1718710485.039|OIDC|rdkafka#producer-1| [thrd:background]: Failed to retrieve OIDC token from "https://IP:https_port/realms/master/protocol/openid-connect/token": SSL certificate problem: unable to get local issuer certificate (-1)
%3|1718710485.039|ERROR|rdkafka#producer-1| [thrd:background]: Failed to acquire SASL OAUTHBEARER token: SSL certificate problem: unable to get local issuer certificate
%3|1718710496.041|OIDC|rdkafka#producer-1| [thrd:background]: Failed to retrieve OIDC token from "https://IP:https_port/realms/master/protocol/openid-connect/token": SSL certificate problem: unable to get local issuer certificate (-1)

• Provide broker log excerpts
• Critical issue