- 
                Notifications
    You must be signed in to change notification settings 
- Fork 99
Open
Description
I'm trying to set my Producer to use Zstd compression, like this:
	// Prepare the configuration
	kafka::Properties props({
		{"bootstrap.servers", {m_BrokerList}}	// The Kafka Brokers
		});
	// The default partitioner is crc_random, but Java decided to use murmur2_random partitioner instead so
	// we need to do the same to be Java compatible
	props.put("partitioner", "murmur2_random");
	// Use Zstd compression for these messages
	props.put("compression.type", "zstd");
	// Intercept logging, so that it logs using our DataDogLogger and not directly to stdout in it's own format
	props.put("log_level", "4");	// Warning level
	props.put("log_cb", KafkaLog);	// Set the logging callback
	// Create a producer instance
	m_KafkaProducer = std::make_shared<kafka::clients::producer::KafkaProducer>(props);
However, this results in the following error in my logs:
Error from Kafka Lib (Level3): KafkaProducer[18f0a03a-ce6be2ba] failed to be initialized with property[compression.type:zstd], result[-1]: Unsupported value "zstd" for configuration property "compression.codec": libzstd not available at build time
It looks like the version of librdkafka used by Modern-cpp-kafka wasn't built with the Zstd library.
Is it possible to enable Zstd compression with Modern-Cpp-Kafka?
I'm guessing I'd need to build librdkafka with Zstd support and use that instead of the version included by Modern-Cpp-Kafka itself?
Thanks
Ben
Metadata
Metadata
Assignees
Labels
No labels