v2.10.0-gr-experimental - Alloc-Free Consumer and Producer API #4
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
marcin-krystianc
commented
Jul 8, 2025
|
|
||
| # Different packaging for tagged vs untagged builds | ||
| if ($env:GITHUB_REF -match '^refs/tags/') { | ||
| $suffix = "gr-experimental" |
| rd_kafka_message* msg) | ||
| { | ||
| this.msg = msg; | ||
| _hdrsPtr = new Lazy<IntPtr>(() => |
There was a problem hiding this comment.
I just realised that this is not alloc-free. Will update it.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This patch introduces experimental allocation-free APIs to enhance performance for message consumption and production.
Key API changes include:
Consumer API:
IConsumer<TKey, TValue>.ConsumeWithCallbackmethod allows for allocation-free message consumption by providing aMessageReaderstruct to a callback. This eliminates heap allocations associated withConsumeResultfor performance-critical scenarios.Producer API:
IProducer<TKey, TValue>.Produceoverload acceptsReadOnlySpan<byte>for message key and value, enabling allocation-free message production.ProducerBuilder<TKey, TValue>.SetAllocFreeDeliveryHandlermethod allows configuring anAllocFreeDeliveryHandlerthat receives delivery reports via aMessageReaderstruct, avoiding allocations for these reports.Performance
Besides eliminating memory allocations, the new API also eliminates almost the entire managed code overhead. The performance of the new API matches bare C performance. This means that the new API is about 50-100% better than the official API.