Releases: rabbitmq/rabbitmq-server
RabbitMQ 4.2.0
RabbitMQ 4.2.0 is a new feature release.
Breaking Changes and Compatibility Notes
Default value for AMQP 1.0 durable field.
Starting with RabbitMQ 4.2, if a sending client omits the header section, RabbitMQ assumes the durable field to be false complying with the AMQP 1.0 spec:
<field name="durable" type="boolean" default="false"/>
AMQP 1.0 apps or client libraries must set the durable field of the header section to true to mark the message as durable.
Team RabbitMQ recommends client libraries to send messages as durable by default.
All AMQP 1.0 client libraries maintained by Team RabbitMQ send messages as durable by default.
Mandatory flag in Direct Reply-To
Starting with RabbitMQ 4.2, if an AMQP 0.9.1 Direct Reply-To responder (RPC server) publishes with the mandatory flag set, then amq.rabbitmq.reply-to.* is treated as a queue.
Whether the requester (RPC client) is still there to consume the reply is not checked at routing time.
In other words, if the responder publishes to only this queue name, then the message will be considered "routed" and RabbitMQ will therefore not send a basic.return.
Very Rarely Used *.cacerts Settings are Removed from rabbitmq.conf
*.cacerts (not to be confused with cacertfile) settings in rabbitmq.conf did not have the expected effect and were removed
to eliminate confusion.
Quorum Queue Metric Changes
Metrics emitted for Ra-based components (quorum queues, Khepri, Stream Coordinator)
have changed. Some metrics were removed, many were added, some changed their names.
Users relying on Prometheus metrics starting with rabbitmq_raft or rabbitmq_detailed_raft
will need to update their dashboards and/or alerts. If you are using the
RabbitMQ-Quorum-Queues-Raft dashboard,
please update it to the latest version for RabbitMQ 4.2 compatibility.
Release Highlights
SQL Filter Expression for Streams
AMQP 1.0 clients can now define SQL-like filter expressions when consuming from streams, enabling server-side message filtering.
RabbitMQ will only dispatch messages that match the provided filter expression, reducing network traffic and client-side processing overhead.
SQL filter expressions are a more powerful alternative to the AMQP Property Filter Expressions introduced in RabbitMQ 4.1.
RabbitMQ implements a subset of AMQP Filter Expressions Version 1.0 Committee Specification Draft 01 Section 6 including support for:
- Comparison operators (
=,!=,<>,>,<,>=,<=) - Logical operators (
AND,OR,NOT) - Arithmetic operators (
+,-,*,/,%) - Special operators (
LIKE,IN,IS NULL) UTCfunction- Access to the properties and application-properties sections
Examples
Simple expression:
header.priority > 4Complex expression:
order_type IN ('premium', 'express') AND
(customer_region LIKE 'EU-%' OR customer_region = 'US-CA') AND
UTC() < properties.absolute-expiry-time AND
NOT cancelledTo learn more, check out the new documentation guide on Stream Filtering.
Pull Request: #14184
Direct Reply-To for AMQP 1.0
RabbitMQ 4.2 adds Direct Reply-To support for AMQP 1.0, alongside the existing AMQP 0.9.1 implementation.
It also works across protocols (e.g., AMQP 1.0 requester with AMQP 0.9.1 responder, or vice versa).
For more information, read our updated documentation on Direct Reply-To.
Pull Request: #14474
New Tooling for More Automated Blue-Green Deployment Migrations from 3.13.x Clusters to 4.2.x
Blue-Green Deployment migration from RabbitMQ 3.13.x
to 4.2.0 is now easier to automate thanks to a new set of commands provided by rabbitmqadmin v2.
Incoming and Outgoing Message Interceptors for Native Protocols (AMQP 1.0, AMQP 0-9-1, MQTTv3, MQTTv5)
Incoming and outgoing messages can now be intercepted on the broker.
This works for AMQP 1.0, AMQP 0.9.1, MQTTv3, and MQTTv5.
What the interceptor does is entirely up to its implementation, for example it can validate message metadata, add annotations, or perform arbitrary side effects.
Custom interceptors can be developed and integrated via plugins.
Two new optional built-in interceptors were added to RabbitMQ:
- Timestamps for outgoing messages
- Setting client ID of publishing MQTT client
Detailed information can be found in the Message Interceptor documentation.
Khepri Enabled by Default for New Clusters
RabbitMQ supports two databases to store
metadata such as virtual hosts,
topology, runtime parameters, policies, internal users and so on: Mnesia and
Khepri. That metadata store is also at the heart of clustering in RabbitMQ. As
of RabbitMQ 4.2.0, Khepri is the default metadata store for new
deployments.
Khepri is based on the
same Raft consensus algorithm used by quorum queues and streams. The goal is to
have a consistent well defined behaviour around all queries and updates of
metadata across an entire cluster, especially when the cluster suffers
increased latency or network issues for instance. It also comes with increased
performance in several use cases, even though this was not a goal.
A new RabbitMQ 4.2.0+ node will use Khepri by default. If you upgrade an
existing node or cluster, it will continue to use whatever metadata store it
was using so far.
If you did not enable Khepri yet, it is recommended that you enable it:
rabbitmqctl enable_feature_flag khepri_db
Khepri will become mandatory in a future minor version. Mnesia support will be
dropped in a future major version. These exact versions are to be decided.
Local Shovels
In addition to AMQP 0-9-1 and AMQP 1.0, Shovels
now support a new "protocol" option called local.
These specialized shovels are internally based on AMQP 1.0 but instead of
separate TCP connections, use the intra-cluster connections
between cluster nodes and the internal API for consumption, publishing
and AMQP 1.0 credit flow.
Such shovels can only be used for consuming and publishing
within the same cluster, not across clusters, but can offer
higher throughput and use fewer resources per connections
than their AMQP 0-9-1 and AMQP 1.0 counterparts.
Upgrading to 4.2.0
Documentation guides on upgrades
See the Upgrading guide for documentation on upgrades and GitHub releases
for release notes of individual releases.
This release series supports upgrades from 4.1.x, 4.0.x and 3.13.x.
If upgrading from a 3.13.x cluster that uses classic mirrored queues,
take a look at what modern CLI tools can offer for such migrations away from classic mirrored queues
via Blue/Green deployments.
Blue/Green Deployment-style upgrades are available for migrations
from RabbitMQ 3.12.x series.
New Required Feature Flags
None. The required feature flag set is the same as in 4.1.x and 4.0.x.
Mixed version cluster compatibility
RabbitMQ 4.2.0 nodes can run alongside 4.1.x and 4.0.x nodes. 4.2.x-specific features can only be made available when all nodes in the cluster upgrade to 4.2.0 or a later patch release in the new series.
While operating in mixed version mode, some aspects of the system may not behave as expected.
Once all nodes are upgraded to 4.1.0, these irregularities will go away.
Mixed version clusters are a mechanism that allows rolling upgrade and are not meant to be run for extended
periods of time (no more than a few hours).
Recommended Post-upgrade Procedures
This version does not require any additional post-upgrade procedures
compared to other versions.
Changes Worth Mentioning
Core Server
Enhancements
-
In clusters with a larger number of quorum queues (say, tens of thousands),
quorum queue leadership transfer is now performed gradually and not all at once.Previously tens of thousands of concurrent leader elections
could result in timeouts and some quorum queues ending up
without an elected leader.GitHub issue: #14401
-
Schema data store (Khepri) read concurrency optimizations that can lead to low doublt digit percent
throughput gains on nodes with larger numbers of cores.GitHub issue: #14530
-
Two new
rabbitmq.conf, settingslog.summarize_process_stateandlog.error_logger_format_depth, can be used
to significantly reduce the amount of queue member (replica) state logged in case of an abnormal termination.
...
RabbitMQ 4.1.5
RabbitMQ 4.1.5 is a maintenance release in the 4.1.x release series.
It is strongly recommended that you read 4.1.0 release notes
in detail if upgrading from a version prior to 4.1.0.
Minimum Supported Erlang Version
RabbitMQ and Erlang/OTP Compatibility Matrix has more details on Erlang version requirements for RabbitMQ.
Nodes will fail to start on older Erlang releases.
Changes Worth Mentioning
Release notes can be found on GitHub at rabbitmq-server/release-notes.
Core Server
Bug Fixes
-
Classic queues could run into a rare message store exception that resuulted in
a loss of a few messages.Special kudos to the contributors who have spent a very significant amount of time
reproducing and debugging the issue: @lhoguin @lukebakken @trvrnrth @gomoripeti -
Messages routed to quorum queues during or immediately before a network partition,
in some cases were not re-republished internally in certain cases.GitHub issue: #14589
-
Quorum queues with disabled poison message handling
(an unlimited number of redeliveries, which is not a recommended practice) could accumulate
a significant number of Raft log segment files.
Streams
Bug Fixes
-
Streams declaration response was missing a leader field when the stream was declared via AMQP 1.0 or 0.9.1
(that is, not the RabbitMQ Stream Protocol).GitHub issue: #14476
CLI Tools
Bug Fixes
-
Fixes JSON formatting of cluster nodes produced by
rabbitmq-diagnostics status --formatter jsonGitHub issue: #14509
Shovel Prometheus Plugin
Bug Fixes
-
Requesting shovel status via Prometheus at a specific moment resulted in an exception
because the plugin did not consider a highly transient starting state.GitHub issue: #14481
RabbitMQ 4.2.0-rc.1
RabbitMQ 4.2.0-rc.1 is a feature release candidate.
Breaking Changes and Compatibility Notes
Default value for AMQP 1.0 durable field.
Starting with RabbitMQ 4.2, if a sending client omits the header section, RabbitMQ assumes the durable field to be false complying with the AMQP 1.0 spec:
<field name="durable" type="boolean" default="false"/>
AMQP 1.0 apps or client libraries must set the durable field of the header section to true to mark the message as durable.
Team RabbitMQ recommends client libraries to send messages as durable by default.
All AMQP 1.0 client libraries maintained by Team RabbitMQ send messages as durable by default.
Mandatory flag in Direct Reply-To
Starting with RabbitMQ 4.2, if an AMQP 0.9.1 Direct Reply-To responder (RPC server) publishes with the mandatory flag set, then amq.rabbitmq.reply-to.* is treated as a queue.
Whether the requester (RPC client) is still there to consume the reply is not checked at routing time.
In other words, if the responder publishes to only this queue name, then the message will be considered "routed" and RabbitMQ will therefore not send a basic.return.
Very Rarely Used *.cacerts Settings are Removed from rabbitmq.conf
*.cacerts (not to be confused with cacertfile) settings in rabbitmq.conf did not have the expected effect and were removed
to eliminate confusion.
Release Highlights
SQL Filter Expression for Streams
AMQP 1.0 clients can now define SQL-like filter expressions when consuming from streams, enabling server-side message filtering.
RabbitMQ will only dispatch messages that match the provided filter expression, reducing network traffic and client-side processing overhead.
SQL filter expressions are a more powerful alternative to the AMQP Property Filter Expressions introduced in RabbitMQ 4.1.
RabbitMQ implements a subset of AMQP Filter Expressions Version 1.0 Committee Specification Draft 01 Section 6 including support for:
- Comparison operators (
=,!=,<>,>,<,>=,<=) - Logical operators (
AND,OR,NOT) - Arithmetic operators (
+,-,*,/,%) - Special operators (
LIKE,IN,IS NULL) UTCfunction- Access to the properties and application-properties sections
Examples
Simple expression:
header.priority > 4Complex expression:
order_type IN ('premium', 'express') AND
(customer_region LIKE 'EU-%' OR customer_region = 'US-CA') AND
UTC() < properties.absolute-expiry-time AND
NOT cancelledTo learn more, check out the new documentation guide on Stream Filtering.
Pull Request: #14184
Direct Reply-To for AMQP 1.0
RabbitMQ 4.2 adds Direct Reply-To support for AMQP 1.0, alongside the existing AMQP 0.9.1 implementation.
It also works across protocols (e.g., AMQP 1.0 requester with AMQP 0.9.1 responder, or vice versa).
For more information, read our updated documentation on Direct Reply-To.
Pull Request: #14474
New Tooling for More Automated Blue-Green Deployment Migrations from 3.13.x Clusters to 4.2.x
Blue-Green Deployment migration from RabbitMQ 3.13.x
to 4.2.0 is now easier to automate thanks to a new set of commands provided by rabbitmqadmin v2.
Incoming and Outgoing Message Interceptors for Native Protocols (AMQP 1.0, AMQP 0-9-1, MQTTv3, MQTTv5)
Incoming and outgoing messages can now be intercepted on the broker.
This works for AMQP 1.0, AMQP 0.9.1, MQTTv3, and MQTTv5.
What the interceptor does is entirely up to its implementation, for example it can validate message metadata, add annotations, or perform arbitrary side effects.
Custom interceptors can be developed and integrated via plugins.
Two new optional built-in interceptors were added to RabbitMQ:
- Timestamps for outgoing messages
- Setting client ID of publishing MQTT client
Detailed information can be found in the Message Interceptor documentation.
Khepri Enabled by Default for New Clusters
RabbitMQ supports two databases to store
metadata such as virtual hosts,
topology, runtime parameters, policies, internal users and so on: Mnesia and
Khepri. That metadata store is also at the heart of clustering in RabbitMQ. As
of RabbitMQ 4.2.0, Khepri is the default metadata store for new
deployments.
Khepri is based on the
same Raft consensus algorithm used by quorum queues and streams. The goal is to
have a consistent well defined behaviour around all queries and updates of
metadata across an entire cluster, especially when the cluster suffers
increased latency or network issues for instance. It also comes with increased
performance in several use cases, even though this was not a goal.
A new RabbitMQ 4.2.0+ node will use Khepri by default. If you upgrade an
existing node or cluster, it will continue to use whatever metadata store it
was using so far.
If you did not enable Khepri yet, it is recommended that you enable it:
rabbitmqctl enable_feature_flag khepri_db
Khepri will become mandatory in a future minor version. Mnesia support will be
dropped in a future major version. These exact versions are to be decided.
Local Shovels
In addition to AMQP 0-9-1 and AMQP 1.0, Shovels
now support a new "protocol" option called local.
These specialized shovels are internally based on AMQP 1.0 but instead of
separate TCP connections, use the intra-cluster connections
between cluster nodes and the internal API for consumption, publishing
and AMQP 1.0 credit flow.
Such shovels can only be used for consuming and publishing
within the same cluster, not across clusters, but can offer
higher throughput and use fewer resources per connections
than their AMQP 0-9-1 and AMQP 1.0 counterparts.
Upgrading to 4.2.0
Documentation guides on upgrades
See the Upgrading guide for documentation on upgrades and GitHub releases
for release notes of individual releases.
This release series supports upgrades from 4.1.x, 4.0.x and 3.13.x.
If upgrading from a 3.13.x cluster that uses classic mirrored queues,
take a look at what modern CLI tools can offer for such migrations away from classic mirrored queues
via Blue/Green deployments.
Blue/Green Deployment-style upgrades are available for migrations
from RabbitMQ 3.12.x series.
New Required Feature Flags
None. The required feature flag set is the same as in 4.1.x and 4.0.x.
Mixed version cluster compatibility
RabbitMQ 4.2.0 nodes can run alongside 4.1.x and 4.0.x nodes. 4.2.x-specific features can only be made available when all nodes in the cluster upgrade to 4.2.0 or a later patch release in the new series.
While operating in mixed version mode, some aspects of the system may not behave as expected.
Once all nodes are upgraded to 4.1.0, these irregularities will go away.
Mixed version clusters are a mechanism that allows rolling upgrade and are not meant to be run for extended
periods of time (no more than a few hours).
Recommended Post-upgrade Procedures
This version does not require any additional post-upgrade procedures
compared to other versions.
Changes Worth Mentioning
Core Server
Enhancements
-
In clusters with a larger number of quorum queues (say, tens of thousands),
quorum queue leadership transfer is now performed gradually and not all at once.Previously tens of thousands of concurrent leader elections
could result in timeouts and some quorum queues ending up
without an elected leader.GitHub issue: #14401
-
Schema data store (Khepri) read concurrency optimizations that can lead to low doublt digit percent
throughput gains on nodes with larger numbers of cores.GitHub issue: #14530
-
Two new
rabbitmq.conf, settingslog.summarize_process_stateandlog.error_logger_format_depth, can be used
to significantly reduce the amount of queue member (replica) state logged in case of an abnormal termination.Limiting logging helps avoid memory allocation spikes.
-
When a configured authentication or authorization backend comes from a known
plugin but the plugin is not enabled, the node will now refuse to start.Previously the node would boot but client connections would fail because
of the missing backend modules.
...
RabbitMQ 4.2.0-beta.4
RabbitMQ 4.2.0-beta.4 is a preview of a feature release.
Breaking Changes and Compatibility Notes
Default value for AMQP 1.0 durable field.
Starting with RabbitMQ 4.2, if a sending client omits the header section, RabbitMQ assumes the durable field to be false complying with the AMQP 1.0 spec:
<field name="durable" type="boolean" default="false"/>
AMQP 1.0 apps or client libraries must set the durable field of the header section to true to mark the message as durable.
Team RabbitMQ recommends client libraries to send messages as durable by default.
All AMQP 1.0 client libraries maintained by Team RabbitMQ send messages as durable by default.
Mandatory flag in Direct Reply-To
Starting with RabbitMQ 4.2, if an AMQP 0.9.1 Direct Reply-To responder (RPC server) publishes with the mandatory flag set, then amq.rabbitmq.reply-to.* is treated as a queue.
Whether the requester (RPC client) is still there to consume the reply is not checked at routing time.
In other words, if the responder publishes to only this queue name, then the message will be considered "routed" and RabbitMQ will therefore not send a basic.return.
Release Highlights
SQL Filter Expression for Streams
AMQP 1.0 clients can now define SQL-like filter expressions when consuming from streams, enabling server-side message filtering.
RabbitMQ will only dispatch messages that match the provided filter expression, reducing network traffic and client-side processing overhead.
SQL filter expressions are a more powerful alternative to the AMQP Property Filter Expressions introduced in RabbitMQ 4.1.
RabbitMQ implements a subset of AMQP Filter Expressions Version 1.0 Committee Specification Draft 01 Section 6 including support for:
- Comparison operators (
=,!=,<>,>,<,>=,<=) - Logical operators (
AND,OR,NOT) - Arithmetic operators (
+,-,*,/,%) - Special operators (
LIKE,IN,IS NULL) UTCfunction- Access to the properties and application-properties sections
Examples
Simple expression:
header.priority > 4Complex expression:
order_type IN ('premium', 'express') AND
(customer_region LIKE 'EU-%' OR customer_region = 'US-CA') AND
UTC() < properties.absolute-expiry-time AND
NOT cancelledTo learn more, check out the new documentation guide on Stream Filtering.
Pull Request: #14184
Direct Reply-To for AMQP 1.0
RabbitMQ 4.2 adds Direct Reply-To support for AMQP 1.0, alongside the existing AMQP 0.9.1 implementation.
It also works across protocols (e.g., AMQP 1.0 requester with AMQP 0.9.1 responder, or vice versa).
For more information, read our updated documentation on Direct Reply-To.
Pull Request: #14474
New Tooling for More Automated Blue-Green Deployment Migrations from 3.13.x Clusters to 4.2.x
Blue-Green Deployment migration from RabbitMQ 3.13.x
to 4.2.0 is now easier to automate thanks to a new set of commands provided by rabbitmqadmin v2.
Incoming and Outgoing Message Interceptors for Native Protocols (AMQP 1.0, AMQP 0-9-1, MQTTv3, MQTTv5)
Incoming and outgoing messages can now be intercepted on the broker.
This works for AMQP 1.0, AMQP 0.9.1, MQTTv3, and MQTTv5.
What the interceptor does is entirely up to its implementation, for example it can validate message metadata, add annotations, or perform arbitrary side effects.
Custom interceptors can be developed and integrated via plugins.
Two new optional built-in interceptors were added to RabbitMQ:
- Timestamps for outgoing messages
- Setting client ID of publishing MQTT client
Detailed information can be found in the Message Interceptor documentation.
Khepri Enabled by Default for New Clusters
RabbitMQ supports two databases to store
metadata such as virtual hosts,
topology, runtime parameters, policies, internal users and so on: Mnesia and
Khepri. That metadata store is also at the heart of clustering in RabbitMQ. As
of RabbitMQ 4.2.0, Khepri is the default metadata store for new
deployments.
Khepri is based on the
same Raft consensus algorithm used by quorum queues and streams. The goal is to
have a consistent well defined behaviour around all queries and updates of
metadata across an entire cluster, especially when the cluster suffers
increased latency or network issues for instance. It also comes with increased
performance in several use cases, even though this was not a goal.
A new RabbitMQ 4.2.0+ node will use Khepri by default. If you upgrade an
existing node or cluster, it will continue to use whatever metadata store it
was using so far.
If you did not enable Khepri yet, it is recommended that you enable it:
rabbitmqctl enable_feature_flag khepri_db
Khepri will become mandatory in a future minor version. Mnesia support will be
dropped in a future major version. These exact versions are to be decided.
Local Shovels
In addition to AMQP 0-9-1 and AMQP 1.0, Shovels
now support a new "protocol" option called local.
These specialized shovels are internally based on AMQP 1.0 but instead of
separate TCP connections, use the intra-cluster connections
between cluster nodes and the internal API for consumption, publishing
and AMQP 1.0 credit flow.
Such shovels can only be used for consuming and publishing
within the same cluster, not across clusters, but can offer
higher throughput and use fewer resources per connections
than their AMQP 0-9-1 and AMQP 1.0 counterparts.
Upgrading to 4.2.0
Documentation guides on upgrades
See the Upgrading guide for documentation on upgrades and GitHub releases
for release notes of individual releases.
This release series supports upgrades from 4.1.x, 4.0.xand3.13.x`.
Blue/Green Deployment-style upgrades are avaialble for migrations
from RabbitMQ 3.12.x series.
New Required Feature Flags
None. The required feature flag set is the same as in 4.1.x and 4.0.x.
Mixed version cluster compatibility
RabbitMQ 4.2.0 nodes can run alongside 4.1.x and 4.0.x nodes. 4.2.x-specific features can only be made available when all nodes in the cluster upgrade to 4.2.0 or a later patch release in the new series.
While operating in mixed version mode, some aspects of the system may not behave as expected.
Once all nodes are upgraded to 4.1.0, these irregularities will go away.
Mixed version clusters are a mechanism that allows rolling upgrade and are not meant to be run for extended
periods of time (no more than a few hours).
Recommended Post-upgrade Procedures
This version does not require any additional post-upgrade procedures
compared to other versions.
Changes Worth Mentioning
This section is incomplete and will be expanded as 4.2 approaches its release candidate stage.
Core Server
Enhancements
-
In clusters with a larger number of quorum queues (say, tens of thousands),
quorum queue leadership transfer is now performed gradually and not all at once.Previously tens of thousands of concurrent leader elections
could result in timeouts and some quorum queues ending up
without an elected leader.GitHub issue: #14401
-
Schema data store (Khepri) read concurrency optimizations that can lead to low doublt digit percent
throughput gains on nodes with larger numbers of cores.GitHub issue: #14530
-
Two new
rabbitmq.conf, settingslog.summarize_process_stateandlog.error_logger_format_depth, can be used
to significantly reduce the amount of queue member (replica) state logged in case of an abnormal termination.Limiting logging helps avoid memory allocation spikes.
-
When a configured authentication or authorization backend comes from a known
plugin but the plugin is not enabled, the node will now refuse to start.Previously the node would boot but client connections would fail because
of the missing backend modules. -
Similarly to the number of queues and virtual hosts, it is now possible
to configure a limit on the cluster-wide number of exchanges that applications
can create:# Applications won't be ...
RabbitMQ 4.2.0-beta.3
RabbitMQ 4.2.0-beta.3 is a preview of a feature release.
Breaking Changes and Compatibility Notes
Default value for AMQP 1.0 durable field.
Starting with RabbitMQ 4.2, if a sending client omits the header section, RabbitMQ assumes the durable field to be false complying with the AMQP 1.0 spec:
<field name="durable" type="boolean" default="false"/>
AMQP 1.0 apps or client libraries must set the durable field of the header section to true to mark the message as durable.
Team RabbitMQ recommends client libraries to send messages as durable by default.
All AMQP 1.0 client libraries maintained by Team RabbitMQ send messages as durable by default.
Mandatory flag in Direct Reply-To
Starting with RabbitMQ 4.2, if an AMQP 0.9.1 Direct Reply-To responder (RPC server) publishes with the mandatory flag set, then amq.rabbitmq.reply-to.* is treated as a queue.
Whether the requester (RPC client) is still there to consume the reply is not checked at routing time.
In other words, if the responder publishes to only this queue name, then the message will be considered "routed" and RabbitMQ will therefore not send a basic.return.
Release Highlights
SQL Filter Expression for Streams
AMQP 1.0 clients can now define SQL-like filter expressions when consuming from streams, enabling server-side message filtering.
RabbitMQ will only dispatch messages that match the provided filter expression, reducing network traffic and client-side processing overhead.
SQL filter expressions are a more powerful alternative to the AMQP Property Filter Expressions introduced in RabbitMQ 4.1.
RabbitMQ implements a subset of AMQP Filter Expressions Version 1.0 Committee Specification Draft 01 Section 6 including support for:
- Comparison operators (
=,!=,<>,>,<,>=,<=) - Logical operators (
AND,OR,NOT) - Arithmetic operators (
+,-,*,/,%) - Special operators (
LIKE,IN,IS NULL) UTCfunction- Access to the properties and application-properties sections
Examples
Simple expression:
header.priority > 4Complex expression:
order_type IN ('premium', 'express') AND
(customer_region LIKE 'EU-%' OR customer_region = 'US-CA') AND
UTC() < properties.absolute-expiry-time AND
NOT cancelledTo learn more, check out the new documentation guide on Stream Filtering.
Pull Request: #14184
Direct Reply-To for AMQP 1.0
RabbitMQ 4.2 adds Direct Reply-To support for AMQP 1.0, alongside the existing AMQP 0.9.1 implementation.
It also works across protocols (e.g., AMQP 1.0 requester with AMQP 0.9.1 responder, or vice versa).
For more information, read our updated documentation on Direct Reply-To.
Pull Request: #14474
New Tooling for More Automated Blue-Green Deployment Migrations from 3.13.x Clusters to 4.2.x
Blue-Green Deployment migration from RabbitMQ 3.13.x
to 4.2.0 is now easier to automate thanks to a new set of commands provided by rabbitmqadmin v2.
Incoming and Outgoing Message Interceptors for Native Protocols
Incoming and outgoing messages can now be intercepted on the broker.
This works for AMQP 1.0, AMQP 0.9.1, and MQTT.
What the interceptor does is entirely up to its implementation, for example it can validate message metadata, add annotations, or perform arbitrary side effects.
Custom interceptors can be developed and integrated via plugins.
Two new optional built-in interceptors were added to RabbitMQ:
- Timestamps for outgoing messages
- Setting client ID of publishing MQTT client
Detailed information can be found in the Message Interceptor documentation.
Khepri Enabled by Default for New Clusters
RabbitMQ supports two databases to store
metadata such as virtual hosts,
topology, runtime parameters, policies, internal users and so on: Mnesia and
Khepri. That metadata store is also at the heart of clustering in RabbitMQ. As
of RabbitMQ 4.2.0, Khepri is the default metadata store for new
deployments.
Khepri is based on the
same Raft consensus algorithm used by quorum queues and streams. The goal is to
have a consistent well defined behaviour around all queries and updates of
metadata across an entire cluster, especially when the cluster suffers
increased latency or network issues for instance. It also comes with increased
performance in several use cases, even though this was not a goal.
A new RabbitMQ 4.2.0+ node will use Khepri by default. If you upgrade an
existing node or cluster, it will continue to use whatever metadata store it
was using so far.
If you did not enable Khepri yet, it is recommended that you enable it:
rabbitmqctl enable_feature_flag khepri_db
Khepri will become mandatory in a future minor version. Mnesia support will be
dropped in a future major version. These exact versions are to be decided.
Local Shovels
In addition to AMQP 0-9-1 and AMQP 1.0, Shovels
now support a new "protocol" option called local.
These specialized shovels are internally based on AMQP 1.0 but instead of
separate TCP connections, use the intra-cluster connections
between cluster nodes and the internal API for consumption, publishing
and AMQP 1.0 credit flow.
Such shovels can only be used for consuming and publishing
within the same cluster, not across clusters, but can offer
higher throughput and use fewer resources per connections
than their AMQP 0-9-1 and AMQP 1.0 counterparts.
Upgrading to 4.2.0
Documentation guides on upgrades
See the Upgrading guide for documentation on upgrades and GitHub releases
for release notes of individual releases.
This release series supports upgrades from 4.1.x, 4.0.xand3.13.x`.
Blue/Green Deployment-style upgrades are avaialble for migrations
from RabbitMQ 3.12.x series.
New Required Feature Flags
None. The required feature flag set is the same as in 4.1.x and 4.0.x.
Mixed version cluster compatibility
RabbitMQ 4.2.0 nodes can run alongside 4.1.x and 4.0.x nodes. 4.2.x-specific features can only be made available when all nodes in the cluster upgrade to 4.2.0 or a later patch release in the new series.
While operating in mixed version mode, some aspects of the system may not behave as expected.
Once all nodes are upgraded to 4.1.0, these irregularities will go away.
Mixed version clusters are a mechanism that allows rolling upgrade and are not meant to be run for extended
periods of time (no more than a few hours).
Recommended Post-upgrade Procedures
This version does not require any additional post-upgrade procedures
compared to other versions.
Changes Worth Mentioning
This section is incomplete and will be expanded as 4.2 approaches its release candidate stage.
Core Server
Enhancements
-
In clusters with a larger number of quorum queues (say, tens of thousands),
quorum queue leadership transfer is now performed gradually and not all at once.Previously tens of thousands of concurrent leader elections
could result in timeouts and some quorum queues ending up
without an elected leader.GitHub issue: #14401
-
Schema data store (Khepri) read concurrency optimizations that can lead to low doublt digit percent
throughput gains on nodes with larger numbers of cores.GitHub issue: #14530
-
Two new
rabbitmq.conf, settingslog.summarize_process_stateandlog.error_logger_format_depth, can be used
to significantly reduce the amount of queue member (replica) state logged in case of an abnormal termination.Limiting logging helps avoid memory allocation spikes.
-
When a configured authentication or authorization backend comes from a known
plugin but the plugin is not enabled, the node will now refuse to start.Previously the node would boot but client connections would fail because
of the missing backend modules. -
Similarly to the number of queues and virtual hosts, it is now possible
to configure a limit on the cluster-wide number of exchanges that applications
can create:# Applications won't be able to declare more than 200 exchanges # (incl...
RabbitMQ 4.1.4
RabbitMQ 4.1.4 is a maintenance release in the 4.1.x release series.
It is strongly recommended that you read 4.1.0 release notes
in detail if upgrading from a version prior to 4.1.0.
Minimum Supported Erlang Version
RabbitMQ and Erlang/OTP Compatibility Matrix has more details on Erlang version requirements for RabbitMQ.
Nodes will fail to start on older Erlang releases.
Changes Worth Mentioning
Release notes can be found on GitHub at rabbitmq-server/release-notes.
Core Server
Bug Fixes
-
default_password,ssl_options.passwordnow can tell between a generated random password
value and an encrypted value better.Encrypted values must be prefixed with
encrypted:. All other values, including
generated passwords that contain a colon (:), will be considered non-encrypted ones.GitHub issue: #14365
-
Import of definition files that contained topic exchange permissions failed.
GitHub issue: #14409
Enhancements
-
When a configured authentication or authorization backend comes from a known
plugin but the plugin is not enabled, the node will now refuse to start.Previously the node would boot but client connections would fail because
of the missing backend modules. -
Similarly to the number of queues and virtual hosts, it is now possible
to configure a limit on the cluster-wide number of exchanges that applications
can create:# Applications won't be able to decalre more than 200 exchanges # (including the protocol-standard pre-declared ones) in the cluster cluster_exchange_limit = 200
The setting must be set to the same value on all cluster nodes.
As with other limits, the goal is to introduce a guard against applications
that leak resources.GitHub issue: #14339
-
In some environments, the use of load balancers is mandated and therefore
Local Random exchanges cannot be used.A new setting,
exchange_types.local_random.enabled, can be used to explicitly disable the local random exchange type,
returning an error to the clients that try to declare such an exchange:# use in environments where the locality requirements of the local random exchange type cannot # be satisfied exchange_types.local_random.enabled = false
GitHub issue: #14363
-
RABBITMQ_MAX_OPEN_FILESis a new environment variable supported by therabbitmq-server
startup script on Linux, macOS and BSD family operating systems.It does not replace the standard mechanisms of increasing the maximum allowed
file handle limit for the node. However, it can be useful in environments where the
soft (current) limit is meaningfully lower than the hard (maximum allowed per system configuration) limit, and the user can only control the soft limit.For example, this is the case in most hosted Kubernetes environments.
Stream Plugin
Bug Fixes
-
Stream client connections that authenticate using a JWT token (OAuth 2) have
to periodically renew their JWT tokens. Should such an update fail,
the RabbitMQ Stream Protocol connection will be immediately closed.In addition, stream connections now verify that the newly obtained JWT
token still grants access to the virtual host the client is connected to. -
If a Single Active Consumer could not leave its SAC group because the stream
coordinator was not available, the operation will now be retried a few times
before reporting a failure back to the client.GitHub issue: #14341
-
Advertised TLS hostname setting value was not passed on to the stream protocol
metadata frame.GitHub issue: #14345
CLI TOols
Bug Fixes
-
rabbitmqctl cluster_status --formatter=jsondouble converted therunning_nodes
field.GitHub issue: #14382
Shovel Plugin
Bug Fixes
-
rabbitmqctl shovel_statuscommand now handles the presence of metrics.GitHub issue: #14298
-
AMQP 1.0 shovels now perform stricter validation on the
src-delete-aftervalues.
They support a narrower set of values compared to their AMQP 0-9-1 counterparts
but shovel definition validation did not account for that difference.GitHub issue: #14420
Dependency Changes
Source Code Archives
To obtain source code of the entire distribution, please download the archive named rabbitmq-server-4.1.4.tar.xz
instead of the source tarball produced by GitHub.
RabbitMQ 4.2.0-beta.2
RabbitMQ 4.2.0-beta.2 is a preview of a feature release.
Breaking Changes and Compatibility Notes
Default value for AMQP 1.0 durable field.
Starting with RabbitMQ 4.2, if a sending client omits the header section, RabbitMQ assumes the durable field to be false complying with the AMQP 1.0 spec:
<field name="durable" type="boolean" default="false"/>
AMQP 1.0 apps or client libraries must set the durable field of the header section to true to mark the message as durable.
Team RabbitMQ recommends client libraries to send messages as durable by default.
All AMQP 1.0 client libraries maintained by Team RabbitMQ send messages as durable by default.
Release Highlights
SQL Filter Expression for Streams
AMQP 1.0 clients can now define SQL-like filter expressions when consuming from streams, enabling server-side message filtering.
RabbitMQ will only dispatch messages that match the provided filter expression, reducing network traffic and client-side processing overhead.
SQL filter expressions are a more powerful alternative to the AMQP Property Filter Expressions introduced in RabbitMQ 4.1.
RabbitMQ implements a subset of AMQP Filter Expressions Version 1.0 Committee Specification Draft 01 Section 6 including support for:
- Comparison operators (
=,!=,<>,>,<,>=,<=) - Logical operators (
AND,OR,NOT) - Arithmetic operators (
+,-,*,/,%) - Special operators (
LIKE,IN,IS NULL) UTCfunction- Access to the properties and application-properties sections
Examples
Simple expression:
header.priority > 4Complex expression:
order_type IN ('premium', 'express') AND
(customer_region LIKE 'EU-%' OR customer_region = 'US-CA') AND
UTC() < properties.absolute-expiry-time AND
NOT cancelledTo learn more, check out the new documentation guide on Stream Filtering.
Pull Request: #14184
New Tooling for More Automated Blue-Green Deployment Migrations from 3.13.x Clusters to 4.2.x
Blue-Green Deployment migration from RabbitMQ 3.13.x
to 4.2.0 is now easier to automate thanks to a new set of commands provided by rabbitmqadmin v2.
Incoming and Outgoing Message Interceptors for Native Protocols
Incoming and outgoing messages can now be intercepted on the broker.
This works for AMQP 1.0, AMQP 0.9.1, and MQTT.
What the interceptor does is entirely up to its implementation, for example it can validate message metadata, add annotations, or perform arbitrary side effects.
Custom interceptors can be developed and integrated via plugins.
Two new optional built-in interceptors were added to RabbitMQ:
- Timestamps for outgoing messages
- Setting client ID of publishing MQTT client
Detailed information can be found in the Message Interceptor documentation.
Khepri Enabled by Default for New Clusters
RabbitMQ supports two databases to store
metadata such as virtual hosts,
topology, runtime parameters, policies, internal users and so on: Mnesia and
Khepri. That metadata store is also at the heart of clustering in RabbitMQ. As
of RabbitMQ 4.2.0, Khepri is the default metadata store for new
deployments.
Khepri is based on the
same Raft consensus algorithm used by quorum queues and streams. The goal is to
have a consistent well defined behaviour around all queries and updates of
metadata across an entire cluster, especially when the cluster suffers
increased latency or network issues for instance. It also comes with increased
performance in several use cases, even though this was not a goal.
A new RabbitMQ 4.2.0+ node will use Khepri by default. If you upgrade an
existing node or cluster, it will continue to use whatever metadata store it
was using so far.
If you did not enable Khepri yet, it is recommended that you enable it:
rabbitmqctl enable_feature_flag khepri_db
Khepri will become mandatory in a future minor version. Mnesia support will be
dropped in a future major version. These exact versions are to be decided.
Local Shovels
In addition to AMQP 0-9-1 and AMQP 1.0, Shovels
now support a new "protocol" option called local.
These specialized shovels are internally based on AMQP 1.0 but instead of
separate TCP connections, use the intra-cluster connections
between cluster nodes and the internal API for consumption, publishing
and AMQP 1.0 credit flow.
Such shovels can only be used for consuming and publishing
within the same cluster, not across clusters, but can offer
higher throughput and use fewer resources per connections
than their AMQP 0-9-1 and AMQP 1.0 counterparts.
Upgrading to 4.2.0
Documentation guides on upgrades
See the Upgrading guide for documentation on upgrades and GitHub releases
for release notes of individual releases.
This release series supports upgrades from 4.1.x, 4.0.xand3.13.x`.
Blue/Green Deployment-style upgrades are avaialble for migrations
from RabbitMQ 3.12.x series.
New Required Feature Flags
None. The required feature flag set is the same as in 4.1.x and 4.0.x.
Mixed version cluster compatibility
RabbitMQ 4.2.0 nodes can run alongside 4.1.x and 4.0.x nodes. 4.2.x-specific features can only be made available when all nodes in the cluster upgrade to 4.2.0 or a later patch release in the new series.
While operating in mixed version mode, some aspects of the system may not behave as expected.
Once all nodes are upgraded to 4.1.0, these irregularities will go away.
Mixed version clusters are a mechanism that allows rolling upgrade and are not meant to be run for extended
periods of time (no more than a few hours).
Recommended Post-upgrade Procedures
This version does not require any additional post-upgrade procedures
compared to other versions.
Changes Worth Mentioning
This section is incomplete and will be expanded as 4.2 approaches its release candidate stage.
Core Server
Enhancements
-
In clusters with a larger number of quorum queues (say, tens of thousands),
quorum queue leadership transfer is now performed gradually and not all at once.Previously tens of thousands of concurrent leader elections
could result in timeouts and some quorum queues ending up
without an elected leader.GitHub issue: #14401
-
When a configured authentication or authorization backend comes from a known
plugin but the plugin is not enabled, the node will now refuse to start.Previously the node would boot but client connections would fail because
of the missing backend modules. -
Similarly to the number of queues and virtual hosts, it is now possible
to configure a limit on the cluster-wide number of exchanges that applications
can create:# Applications won't be able to decalre more than 200 exchanges # (including the protocol-standard pre-declared ones) in the cluster cluster_exchange_limit = 200
Bug Fixes
-
default_password,ssl_options.passwordnow can tell between a generated random password
value and an encrypted value better.Encrypted values must be prefixed with
encrypted:. All other values, including
generated passwords that contain a colon (:), will be considered non-encrypted ones.GitHub issue: #14365
-
Import of definition files that contained topic exchange permissions failed.
GitHub issue: #14409
Stream Plugin
Bug Fixes
-
Stream client connections that authenticate using a JWT token (OAuth 2) have
to periodically renew their JWT tokens. Should such an update fail,
the RabbitMQ Stream Protocol connection will be immediately closed.In addition, stream connections now verify that the newly obtained JWT
token still grants access to the virtual host the client is connected to.
Dependency Changes
RabbitMQ 4.2.0-beta.1
RabbitMQ 4.2.0-beta.1 is a preview of a feature release.
Breaking Changes and Compatibility Notes
Default value for AMQP 1.0 durable field.
Starting with RabbitMQ 4.2, if a sending client omits the header section, RabbitMQ assumes the durable field to be false complying with the AMQP 1.0 spec:
<field name="durable" type="boolean" default="false"/>
AMQP 1.0 apps or client libraries must set the durable field of the header section to true to mark the message as durable.
Team RabbitMQ recommends client libraries to send messages as durable by default.
All AMQP 1.0 client libraries maintained by Team RabbitMQ send messages as durable by default.
Release Highlights
SQL Filter Expression for Streams
AMQP 1.0 clients can now define SQL-like filter expressions when consuming from streams, enabling server-side message filtering.
RabbitMQ will only dispatch messages that match the provided filter expression, reducing network traffic and client-side processing overhead.
SQL filter expressions are a more powerful alternative to the AMQP Property Filter Expressions introduced in RabbitMQ 4.1.
RabbitMQ implements a subset of AMQP Filter Expressions Version 1.0 Committee Specification Draft 01 Section 6 including support for:
- Comparison operators (
=,!=,<>,>,<,>=,<=) - Logical operators (
AND,OR,NOT) - Arithmetic operators (
+,-,*,/,%) - Special operators (
LIKE,IN,IS NULL) UTCfunction- Access to the properties and application-properties sections
Examples
Simple expression:
header.priority > 4Complex expression:
order_type IN ('premium', 'express') AND
(customer_region LIKE 'EU-%' OR customer_region = 'US-CA') AND
UTC() < properties.absolute-expiry-time AND
NOT cancelledTo learn more, check out the new documentation guide on Stream Filtering.
Pull Request: #14184
New Tooling for More Automated Blue-Green Deployment Migrations from 3.13.x Clusters to 4.2.x
Blue-Green Deployment migration from RabbitMQ 3.13.x
to 4.2.0 is now easier to automate thanks to a new set of commands provided by rabbitmqadmin v2.
Incoming and Outgoing Message Interceptors for Native Protocols
Incoming and outgoing messages can now be intercepted on the broker.
This works for AMQP 1.0, AMQP 0.9.1, and MQTT.
What the interceptor does is entirely up to its implementation, for example it can validate message metadata, add annotations, or perform arbitrary side effects.
Custom interceptors can be developed and integrated via plugins.
Two new optional built-in interceptors were added to RabbitMQ:
- Timestamps for outgoing messages
- Setting client ID of publishing MQTT client
Detailed information can be found in the Message Interceptor documentation.
Khepri Enabled by Default for New Clusters
RabbitMQ supports two databases to store
metadata such as virtual hosts,
topology, runtime parameters, policies, internal users and so on: Mnesia and
Khepri. That metadata store is also at the heart of clustering in RabbitMQ. As
of RabbitMQ 4.2.0, Khepri is the default metadata store for new
deployments.
Khepri is based on the
same Raft consensus algorithm used by quorum queues and streams. The goal is to
have a consistent well defined behaviour around all queries and updates of
metadata across an entire cluster, especially when the cluster suffers
increased latency or network issues for instance. It also comes with increased
performance in several use cases, even though this was not a goal.
A new RabbitMQ 4.2.0+ node will use Khepri by default. If you upgrade an
existing node or cluster, it will continue to use whatever metadata store it
was using so far.
If you did not enable Khepri yet, it is recommended you enable
it:
rabbitmqctl enable_feature_flag khepri_db
Khepri will become mandatory in a future minor version. Mnesia support will be
dropped in a future major version. These exact versions are to be decided.
RabbitMQ 4.1.3
RabbitMQ 4.1.3 is a maintenance release in the 4.1.x release series.
It is strongly recommended that you read 4.1.0 release notes
in detail if upgrading from a version prior to 4.1.0.
Minimum Supported Erlang Version
RabbitMQ and Erlang/OTP Compatibility Matrix has more details on Erlang version requirements for RabbitMQ.
Nodes will fail to start on older Erlang releases.
Changes Worth Mentioning
Release notes can be found on GitHub at rabbitmq-server/release-notes.
Core Server
Bug Fixes
-
Fixes a rare file descriptor leak by quorum queues.
GitHub issue: rabbitmq/ra#553
-
Fixed two exceptions around IEEE 754-2008 serialization in the AMQP 1.0 implementation.
GitHub issue: #14213
-
Queues originally defined before
3.8.0could run into an exception
when emitting metrics.GitHub issue: #14214
-
Avoids a potential classic queue exception logged when the queue was deleted.
GitHub issue: #14276
CLI Tools
Bug Fixes
-
rabbitmq-diagnostics statusproduced a stray line of debug output.GitHub issue: #14266
Enhancements
-
rabbitmqctl list_deprecated_featuresnow lists feature state (permitted,denied) in the target cluster.GitHub issue: #14229
-
Optimizes
rabbitmqctl list_queuesinvocations that only request queue names.GitHub issue: #14281
Dependency Changes
rawas upgraded to2.16.12
Source Code Archives
To obtain source code of the entire distribution, please download the archive named rabbitmq-server-4.1.3.tar.xz
instead of the source tarball produced by GitHub.
RabbitMQ 4.1.2
RabbitMQ 4.1.2 is a maintenance release in the 4.1.x release series.
It is strongly recommended that you read 4.1.0 release notes
in detail if upgrading from a version prior to 4.1.0.
Minimum Supported Erlang Version
RabbitMQ and Erlang/OTP Compatibility Matrix has more details on Erlang version requirements for RabbitMQ.
Nodes will fail to start on older Erlang releases.
Changes Worth Mentioning
Release notes can be found on GitHub at rabbitmq-server/release-notes.
Core Server
Bug Fixes
-
Channels that had consumers that consumed from quorum queues could leak file handles
when those queues were deleted. -
Classic queues now retry opening files when flushing buffers to significantly reduce the probability of running into
eaccesfile system operation errors from the Windows kernel.GitHub issue: #14131
-
A higher priority quorum queue SAC consumer was never activated when a quiescing consumer
returned or requeued its last outstanding delivery.GitHub issue: #14149
Enhancements
-
An opt-in setting that makes a node refuse to boot if there's evidence that the node might have been reset
in the past.GitHub issue: #14125
-
Minor memory footprint optimizations.
Stream Plugin
Bug Fixes
-
Multiple stream and stream SAC (Single Active Consumer) coordinator resilience improvements.
-
Consumer on a stream that was deleted could leak a file descriptor.
GitHub issue: #14143
-
When a stream member (replica) failed, consumer cleanup could affect consumers connected to different nodes.
GitHub issue: #13961
-
Unhandled stream coordinator exceptions now close stream connections, giving clients a chance to reconnect and reset stream state.
GitHub issue: #14040
-
A closing connection could log a scary looking harmless exception.
GitHub issue: #14128
CLI Tools
Enhancements
-
rabbitmq-streams activate_stream_consumeris a new CLI command that forcefully activates a specific consumer in a SAC (Single Active Consumer) group:rabbitmq-streams activate_stream_consumer --stream [stream name] --reference [reference]
This is an emergency operations command that won't be necessary most of the time.
GitHub issue: #14055
Bug Fixes
-
rabbitmq-streams coordinator_statuscommand reported an incorrect Raft machine version.GitHub issue: #14112
-
rabbitmq-diagnostics environmentnow correctly transforms its output
when--formatter=jsonis used.GitHub issue: #14118
Management Plugin
Bug Fixes
-
Eliminated a JavaScript exception that could affect those upgrading from
3.13.xto4.xversions.GitHub issue: #13973
-
Virtual host restart form now sanitizes virtual host name.
Dependency Changes
rawas upgraded to2.16.11redbugwas upgraded to2.1.0
Source Code Archives
To obtain source code of the entire distribution, please download the archive named rabbitmq-server-4.1.2.tar.xz
instead of the source tarball produced by GitHub.