Skip to content

Commit ac5a024

Browse files
michaelklishinmichaelklishin
authored
Merge pull request #2315 from rabbitmq/clients_subbatch_compressions
Add clients compressions implementations
2 parents 953c034 + 272708a commit ac5a024

File tree

12 files changed

+112
-29
lines changed

12 files changed

+112
-29
lines changed

‎docs/stream-core-plugin-comparison.md‎

Lines changed: 26 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -19,24 +19,24 @@ See the License for the specific language governing permissions and
1919
limitations under the License.
2020
-->
2121

22-
# Stream Core vs Stream Plugin
22+
# RabbitMQ Core vs Stream Plugin
2323

2424
## Overview {#overview}
2525

26-
This section covers the differences between [stream core](./streams) and the [stream plugin](./stream).
26+
This section covers the differences between [RabbitMQ core](./streams) and the [stream plugin](./stream).
2727
Stream core designates stream features in the broker with only default plugins activated and through protocols like AMQP 0.9.1, AMQP 1.0, MQTT, and STOMP.
2828

2929

3030
### Feature Matrix
3131

32-
|Feature | Stream Core | Stream Plugin |
32+
|Feature | RabbitMQ Core | Stream Plugin |
3333
|-| ------------------------ | ------------- |
3434
|Activation| Built-in | [Must be activated](./stream#enabling-plugin) |
3535
|Protocol| AMQP 0.9.1 and AMQP 1.0 | [RabbitMQ Stream](https://github.com/rabbitmq/rabbitmq-server/blob/v3.12.x/deps/rabbitmq_stream/docs/PROTOCOL.adoc) |
3636
|Clients| AMQP 0.9.1 clients ([documentation](./streams#usage)). AMQP 1.0 clients ([documentation](/client-libraries/amqp-client-libraries#support-for-streams)) |[RabbitMQ stream clients](./stream#overview) |
3737
|Port| 5672 | 5552 |
3838
|Format| Server-side AMQP 1.0 message format encoding and decoding | Client-side AMQP 1.0 message format encoding and decoding |
39-
|Sub-entry batching| Not supported | Supported ([Java example](https://rabbitmq.github.io/rabbitmq-stream-java-client/snapshot/htmlsingle/#sub-entry-batching-and-compression)) |
39+
|Sub-entry batching| Supported (uncompressed) | Supported ([Java example](https://rabbitmq.github.io/rabbitmq-stream-java-client/snapshot/htmlsingle/#sub-entry-batching-and-compression)). [Clients Compression Options](#stream-clients-sub-entry-batching-compression-options) |
4040
|Offset tracking| Use external store | Built-in server-side support ([Java example](https://rabbitmq.github.io/rabbitmq-stream-java-client/snapshot/htmlsingle/#consumer-offset-tracking)) or external store |
4141
|Publishing deduplication|Not supported | Supported ([Java example](https://rabbitmq.github.io/rabbitmq-stream-java-client/snapshot/htmlsingle/#outbound-message-deduplication)) |
4242
|[Super stream](/blog/2022/07/13/rabbitmq-3-11-feature-preview-super-streams) |Not supported | Supported |
@@ -50,9 +50,30 @@ Streams store messages using the AMQP 1.0 message format.
5050

5151
* RabbitMQ Stream client libraries are expected to support the AMQP 1.0 message format
5252
* The broker handles the [conversion](./conversions) between AMQP 1.0 and AMQP 0.9.1 for AMQP 0.9.1 clients
53-
* AMQP 0.9.1 and stream clients can write to and read from the same stream, but [Sub-Entry Batching](https://rabbitmq.github.io/rabbitmq-stream-java-client/snapshot/htmlsingle/#sub-entry-batching-and-compression) is not supported.
53+
* AMQP 0.9.1,AMQP 1.0 and stream clients can write to and read from the same stream. [Sub-Entry Batching](https://rabbitmq.github.io/rabbitmq-stream-java-client/snapshot/htmlsingle/#sub-entry-batching-and-compression) is supported without compression.
5454
* RabbitMQ Stream supports the following section of the AMQP 1.0 message format:
5555
* properties
5656
* application properties
5757
* application data
5858
* message annotations
59+
60+
### Stream Clients Sub-entry Batching Compression Options
61+
62+
The compression happens client-side. The available compression implementations:
63+
- No compression
64+
- Gzip
65+
- Snappy
66+
- LZ4
67+
- Zstd
68+
69+
The table below explains what options are supported by a number of RabbitMQ Stream Protocol clients:
70+
71+
72+
|Client | Supported | No Compression|Gzip| Snappy | LZ4 | Zstd |
73+
|-| ------------------------ | ------------- |--- | --- | --- |--- |
74+
|[Java](https://github.com/rabbitmq/rabbitmq-stream-java-client)| ✅| ✅ |✅ |✅ |✅ | ✅ |
75+
|[.NET](https://github.com/rabbitmq/rabbitmq-stream-dotnet-client)| ✅| ✅ | ✅ |✅ [via interface](https://rabbitmq.github.io/rabbitmq-stream-dotnet-client/stable/htmlsingle/index.html#sub-entry-batching-and-compression) |✅ [via interface](https://rabbitmq.github.io/rabbitmq-stream-dotnet-client/stable/htmlsingle/index.html#sub-entry-batching-and-compression) | ✅ [via interface](https://rabbitmq.github.io/rabbitmq-stream-dotnet-client/stable/htmlsingle/index.html#sub-entry-batching-and-compression) |
76+
|[Go](https://github.com/rabbitmq/rabbitmq-stream-go-client)| ✅| ✅ |✅ |✅ |✅ | ✅ |
77+
|[Python](https://github.com/rabbitmq-community/rstream)| ✅| ✅ | ✅ |✅ [via interface](https://github.com/rabbitmq-community/rstream?tab=readme-ov-file#sub-entry-batching-and-compression) |✅ [via interface](https://github.com/rabbitmq-community/rstream?tab=readme-ov-file#sub-entry-batching-and-compression) | ✅ [via interface](https://github.com/rabbitmq-community/rstream?tab=readme-ov-file#sub-entry-batching-and-compression) |
78+
|[NodeJS](https://github.com/coders51/rabbitmq-stream-js-client)| ✅| ✅ |✅ | ❌ |❌ | ❌ |
79+
|[Rust](https://github.com/rabbitmq/rabbitmq-stream-rust-client)| ❌| ❌ |❌ |❌ | ❌ |

‎docs/stream.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@ This feature is available in all [currently maintained release series](/release-
3030
Streams can be used as a regular AMQP 0.9.1 queue or through a
3131
[dedicated binary protocol](https://github.com/rabbitmq/rabbitmq-server/blob/v3.12.x/deps/rabbitmq_stream/docs/PROTOCOL.adoc)
3232
plugin and associated client(s).
33-
Please see the [stream core and stream plugin comparison page](./stream-core-plugin-comparison) for the feature matrix.
33+
Please see the [RabbitMQ core and stream plugin comparison page](./stream-core-plugin-comparison) for the feature matrix.
3434

3535
This page covers the Stream plugin, which allows to interact with streams using this
3636
[new binary protocol](https://github.com/rabbitmq/rabbitmq-server/blob/v3.12.x/deps/rabbitmq_stream/docs/PROTOCOL.adoc).

‎docs/streams.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -50,7 +50,7 @@ To answer these questions, streams were not introduced to replace queues but to
5050

5151
The following information details streams usage, and the administration and maintenance operations for streams.
5252

53-
You should also review the [stream plugin](./stream) information to learn more about the usage of streams with the binary RabbitMQ Stream protocol and the [stream core and stream plugin comparison page](./stream-core-plugin-comparison) for the feature matrix.
53+
You should also review the [stream plugin](./stream) information to learn more about the usage of streams with the binary RabbitMQ Stream protocol and the [RabbitMQ core and stream plugin comparison page](./stream-core-plugin-comparison) for the feature matrix.
5454

5555
### Use Cases for Using Streams {#use-cases}
5656

‎versioned_docs/version-3.13/stream-core-plugin-comparison.md‎

Lines changed: 26 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -19,24 +19,24 @@ See the License for the specific language governing permissions and
1919
limitations under the License.
2020
-->
2121

22-
# Stream Core vs Stream Plugin
22+
# RabbitMQ Core vs Stream Plugin
2323

2424
## Overview {#overview}
2525

26-
This section covers the differences between [stream core](./streams) and the [stream plugin](./stream).
26+
This section covers the differences between [RabbitMQ core](./streams) and the [stream plugin](./stream).
2727
Stream core designates stream features in the broker with only default plugins activated and through protocols like AMQP 0.9.1, MQTT, and STOMP.
2828

2929

3030
### Feature Matrix
3131

32-
|Feature | Stream Core | Stream Plugin |
32+
|Feature | RabbitMQ Core | Stream Plugin |
3333
|-| ------------------------ | ------------- |
3434
|Activation| Built-in | [Must be activated](./stream#enabling-plugin) |
3535
|Protocol| AMQP 0.9.1 | [RabbitMQ Stream](https://github.com/rabbitmq/rabbitmq-server/blob/v3.12.x/deps/rabbitmq_stream/docs/PROTOCOL.adoc) |
3636
|Clients| AMQP 0.9.1 clients ([documentation](./streams#usage)) |[RabbitMQ stream clients](./stream#overview) |
3737
|Port| 5672 | 5552 |
3838
|Format| Server-side AMQP 1.0 message format encoding and decoding | Client-side AMQP 1.0 message format encoding and decoding |
39-
|Sub-entry batching| Not supported | Supported ([Java example](https://rabbitmq.github.io/rabbitmq-stream-java-client/snapshot/htmlsingle/#sub-entry-batching-and-compression)) |
39+
|Sub-entry batching| Supported (uncompressed) | Supported ([Java example](https://rabbitmq.github.io/rabbitmq-stream-java-client/snapshot/htmlsingle/#sub-entry-batching-and-compression)). [Clients Compression Options](#stream-clients-sub-entry-batching-compression-options) |
4040
|Offset tracking| Use external store | Built-in server-side support ([Java example](https://rabbitmq.github.io/rabbitmq-stream-java-client/snapshot/htmlsingle/#consumer-offset-tracking)) or external store |
4141
|Publishing deduplication|Not supported | Supported ([Java example](https://rabbitmq.github.io/rabbitmq-stream-java-client/snapshot/htmlsingle/#outbound-message-deduplication)) |
4242
|[Super stream](/blog/2022/07/13/rabbitmq-3-11-feature-preview-super-streams) |Not supported | Supported |
@@ -49,10 +49,30 @@ Stream core designates stream features in the broker with only default plugins a
4949
Streams store messages using the AMQP 1.0 message format.
5050

5151
* RabbitMQ Stream client libraries are expected to support the AMQP 1.0 message format
52-
* The broker handles the conversion between AMQP 1.0 and AMQP 0.9.1 for AMQP 0.9.1 clients
53-
* AMQP 0.9.1 and stream clients can write to and read from the same stream, but [Sub-Entry Batching](https://rabbitmq.github.io/rabbitmq-stream-java-client/snapshot/htmlsingle/#sub-entry-batching-and-compression) is not supported.
52+
* The broker handles the [conversion](./conversions) between AMQP 1.0 and AMQP 0.9.1 for AMQP 0.9.1 clients
53+
* AMQP 0.9.1,AMQP 1.0 and stream clients can write to and read from the same stream. [Sub-Entry Batching](https://rabbitmq.github.io/rabbitmq-stream-java-client/snapshot/htmlsingle/#sub-entry-batching-and-compression) is supported without compression.
5454
* RabbitMQ Stream supports the following section of the AMQP 1.0 message format:
5555
* properties
5656
* application properties
5757
* application data
5858
* message annotations
59+
60+
### Stream Clients Sub-entry Batching Compression Options
61+
62+
The compression happens client-side. The available compression implementations:
63+
- No compression
64+
- Gzip
65+
- Snappy
66+
- LZ4
67+
- Zstd
68+
69+
The table below explains what options are supported by a number of RabbitMQ Stream Protocol clients:
70+
71+
|Client | Supported | No Compression|Gzip| Snappy | LZ4 | Zstd |
72+
|-| ------------------------ | ------------- |--- | --- | --- |--- |
73+
|[Java](https://github.com/rabbitmq/rabbitmq-stream-java-client)| ✅| ✅ |✅ |✅ |✅ | ✅ |
74+
|[.NET](https://github.com/rabbitmq/rabbitmq-stream-dotnet-client)| ✅| ✅ | ✅ |✅ [via interface](https://rabbitmq.github.io/rabbitmq-stream-dotnet-client/stable/htmlsingle/index.html#sub-entry-batching-and-compression) |✅ [via interface](https://rabbitmq.github.io/rabbitmq-stream-dotnet-client/stable/htmlsingle/index.html#sub-entry-batching-and-compression) | ✅ [via interface](https://rabbitmq.github.io/rabbitmq-stream-dotnet-client/stable/htmlsingle/index.html#sub-entry-batching-and-compression) |
75+
|[Go](https://github.com/rabbitmq/rabbitmq-stream-go-client)| ✅| ✅ |✅ |✅ |✅ | ✅ |
76+
|[Python](https://github.com/rabbitmq-community/rstream)| ✅| ✅ | ✅ |✅ [via interface](https://github.com/rabbitmq-community/rstream?tab=readme-ov-file#sub-entry-batching-and-compression) |✅ [via interface](https://github.com/rabbitmq-community/rstream?tab=readme-ov-file#sub-entry-batching-and-compression) | ✅ [via interface](https://github.com/rabbitmq-community/rstream?tab=readme-ov-file#sub-entry-batching-and-compression) |
77+
|[NodeJS](https://github.com/coders51/rabbitmq-stream-js-client)| ✅| ✅ |✅ | ❌ |❌ | ❌ |
78+
|[Rust](https://github.com/rabbitmq/rabbitmq-stream-rust-client)| ❌| ❌ |❌ |❌ | ❌ |

‎versioned_docs/version-3.13/stream.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@ This feature is available in all [currently maintained release series](/release-
3030
Streams can be used as a regular AMQP 0.9.1 queue or through a
3131
[dedicated binary protocol](https://github.com/rabbitmq/rabbitmq-server/blob/v3.12.x/deps/rabbitmq_stream/docs/PROTOCOL.adoc)
3232
plugin and associated client(s).
33-
Please see the [stream core and stream plugin comparison page](./stream-core-plugin-comparison) for the feature matrix.
33+
Please see the [RabbitMQ core and stream plugin comparison page](./stream-core-plugin-comparison) for the feature matrix.
3434

3535
This page covers the Stream plugin, which allows to interact with streams using this
3636
[new binary protocol](https://github.com/rabbitmq/rabbitmq-server/blob/v3.12.x/deps/rabbitmq_stream/docs/PROTOCOL.adoc).

‎versioned_docs/version-3.13/streams.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -50,7 +50,7 @@ To answer these questions, streams were not introduced to replace queues but to
5050

5151
The following information details streams usage, and the administration and maintenance operations for streams.
5252

53-
You should also review the [stream plugin](./stream) information to learn more about the usage of streams with the binary RabbitMQ Stream protocol and the [stream core and stream plugin comparison page](./stream-core-plugin-comparison) for the feature matrix.
53+
You should also review the [stream plugin](./stream) information to learn more about the usage of streams with the binary RabbitMQ Stream protocol and the [RabbitMQ core and stream plugin comparison page](./stream-core-plugin-comparison) for the feature matrix.
5454

5555
### Use Cases for Using Streams {#use-cases}
5656

‎versioned_docs/version-4.0/stream-core-plugin-comparison.md‎

Lines changed: 26 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -19,24 +19,24 @@ See the License for the specific language governing permissions and
1919
limitations under the License.
2020
-->
2121

22-
# Stream Core vs Stream Plugin
22+
# RabbitMQ Core vs Stream Plugin
2323

2424
## Overview {#overview}
2525

26-
This section covers the differences between [stream core](./streams) and the [stream plugin](./stream).
26+
This section covers the differences between [RabbitMQ core](./streams) and the [stream plugin](./stream).
2727
Stream core designates stream features in the broker with only default plugins activated and through protocols like AMQP 0.9.1, AMQP 1.0, MQTT, and STOMP.
2828

2929

3030
### Feature Matrix
3131

32-
|Feature | Stream Core | Stream Plugin |
32+
|Feature | RabbitMQ Core | Stream Plugin |
3333
|-| ------------------------ | ------------- |
3434
|Activation| Built-in | [Must be activated](./stream#enabling-plugin) |
3535
|Protocol| AMQP 0.9.1 and AMQP 1.0 | [RabbitMQ Stream](https://github.com/rabbitmq/rabbitmq-server/blob/v3.12.x/deps/rabbitmq_stream/docs/PROTOCOL.adoc) |
3636
|Clients| AMQP 0.9.1 clients ([documentation](./streams#usage)). AMQP 1.0 clients ([documentation](/client-libraries/amqp-client-libraries#support-for-streams))|[RabbitMQ stream clients](./stream#overview) |
3737
|Port| 5672 | 5552 |
3838
|Format| Server-side AMQP 1.0 message format encoding and decoding | Client-side AMQP 1.0 message format encoding and decoding |
39-
|Sub-entry batching| Not supported | Supported ([Java example](https://rabbitmq.github.io/rabbitmq-stream-java-client/snapshot/htmlsingle/#sub-entry-batching-and-compression)) |
39+
|Sub-entry batching| Supported (uncompressed) | Supported ([Java example](https://rabbitmq.github.io/rabbitmq-stream-java-client/snapshot/htmlsingle/#sub-entry-batching-and-compression)). [Clients Compression Options](#stream-clients-sub-entry-batching-compression-options) |
4040
|Offset tracking| Use external store | Built-in server-side support ([Java example](https://rabbitmq.github.io/rabbitmq-stream-java-client/snapshot/htmlsingle/#consumer-offset-tracking)) or external store |
4141
|Publishing deduplication|Not supported | Supported ([Java example](https://rabbitmq.github.io/rabbitmq-stream-java-client/snapshot/htmlsingle/#outbound-message-deduplication)) |
4242
|[Super stream](/blog/2022/07/13/rabbitmq-3-11-feature-preview-super-streams) |Not supported | Supported |
@@ -50,9 +50,30 @@ Streams store messages using the AMQP 1.0 message format.
5050

5151
* RabbitMQ Stream client libraries are expected to support the AMQP 1.0 message format
5252
* The broker handles the [conversion](./conversions) between AMQP 1.0 and AMQP 0.9.1 for AMQP 0.9.1 clients
53-
* AMQP 0.9.1 and stream clients can write to and read from the same stream, but [Sub-Entry Batching](https://rabbitmq.github.io/rabbitmq-stream-java-client/snapshot/htmlsingle/#sub-entry-batching-and-compression) is not supported.
53+
* AMQP 0.9.1,AMQP 1.0 and stream clients can write to and read from the same stream. [Sub-Entry Batching](https://rabbitmq.github.io/rabbitmq-stream-java-client/snapshot/htmlsingle/#sub-entry-batching-and-compression) is supported without compression.
5454
* RabbitMQ Stream supports the following section of the AMQP 1.0 message format:
5555
* properties
5656
* application properties
5757
* application data
5858
* message annotations
59+
60+
### Stream Clients Sub-entry Batching Compression Options
61+
62+
The compression happens client-side. The available compression implementations:
63+
- No compression
64+
- Gzip
65+
- Snappy
66+
- LZ4
67+
- Zstd
68+
69+
The table below explains what options are supported by a number of RabbitMQ Stream Protocol clients:
70+
71+
72+
|Client | Supported | No Compression|Gzip| Snappy | LZ4 | Zstd |
73+
|-| ------------------------ | ------------- |--- | --- | --- |--- |
74+
|[Java](https://github.com/rabbitmq/rabbitmq-stream-java-client)| ✅| ✅ |✅ |✅ |✅ | ✅ |
75+
|[.NET](https://github.com/rabbitmq/rabbitmq-stream-dotnet-client)| ✅| ✅ | ✅ |✅ [via interface](https://rabbitmq.github.io/rabbitmq-stream-dotnet-client/stable/htmlsingle/index.html#sub-entry-batching-and-compression) |✅ [via interface](https://rabbitmq.github.io/rabbitmq-stream-dotnet-client/stable/htmlsingle/index.html#sub-entry-batching-and-compression) | ✅ [via interface](https://rabbitmq.github.io/rabbitmq-stream-dotnet-client/stable/htmlsingle/index.html#sub-entry-batching-and-compression) |
76+
|[Go](https://github.com/rabbitmq/rabbitmq-stream-go-client)| ✅| ✅ |✅ |✅ |✅ | ✅ |
77+
|[Python](https://github.com/rabbitmq-community/rstream)| ✅| ✅ | ✅ |✅ [via interface](https://github.com/rabbitmq-community/rstream?tab=readme-ov-file#sub-entry-batching-and-compression) |✅ [via interface](https://github.com/rabbitmq-community/rstream?tab=readme-ov-file#sub-entry-batching-and-compression) | ✅ [via interface](https://github.com/rabbitmq-community/rstream?tab=readme-ov-file#sub-entry-batching-and-compression) |
78+
|[NodeJS](https://github.com/coders51/rabbitmq-stream-js-client)| ✅| ✅ |✅ | ❌ |❌ | ❌ |
79+
|[Rust](https://github.com/rabbitmq/rabbitmq-stream-rust-client)| ❌| ❌ |❌ |❌ | ❌ |

‎versioned_docs/version-4.0/stream.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@ This feature is available in all [currently maintained release series](/release-
3030
Streams can be used as a regular AMQP 0.9.1 queue or through a
3131
[dedicated binary protocol](https://github.com/rabbitmq/rabbitmq-server/blob/v3.12.x/deps/rabbitmq_stream/docs/PROTOCOL.adoc)
3232
plugin and associated client(s).
33-
Please see the [stream core and stream plugin comparison page](./stream-core-plugin-comparison) for the feature matrix.
33+
Please see the [RabbitMQ core and stream plugin comparison page](./stream-core-plugin-comparison) for the feature matrix.
3434

3535
This page covers the Stream plugin, which allows to interact with streams using this
3636
[new binary protocol](https://github.com/rabbitmq/rabbitmq-server/blob/v3.12.x/deps/rabbitmq_stream/docs/PROTOCOL.adoc).

0 commit comments

Comments
 (0)