Should it be possible to override a previously set configuration setting with multiple RABBITMQ_CONFIG_FILES?

[warappa]

Describe the bug

Issue

I have the files 10-defaults.conf and 20-tls.conf.
10-defaults.conf enables default port (15672), and
20-tls.conf configures the TLS port (15671)

For local development I don't need TLS and just want to go with simple HTTP endpoint (15672).
For productive, I want to use TLS with 15671, but disable the non-TLS one (15672).

My Dockerfile gets an argument which deletes the 20-tls.conf file if TLS is not desired.

For the messaging itself, you can specify listeners.tcp = none and be done.
For the management tcp settings I wanted to do the same, but RabbitMQ yields an error about "none" not being an integer (obviously).

Workaround

The only workaround I found was to set the ip to localhost (management.tcp.ip = 127.0.0.1). It's pragmatic, but not entirely clean.

Reproduction steps

Have 2 files and RABBITMQ_CONFIG_FILES pointing to the containing directory:

• 10-defaults.conf which enables management on port 15672 and
• 20-tls.conf which tries to disable the non-TLS endpoint for management UI.

In 20-tls.conf, try to disable the HTTP endpoint:

• unset the setting management.tcp.port with the value none - it will not work as RabbitMQ crashes on startup, because it cannot translate the values.
• management.tcp is entirely unsupported ("Did you mean ?") and
• management.tcp.ip really wants to have an IP

Expected behavior

Management UI non-TLS (HTTP) endpoint should be as easily be disableable as the actual messaging port (listeners.tcp) with none.

Additional context

Investigation

There seems to be a mapping that lets listeners.tcp be configured with none.

rabbitmq-server/deps/rabbit/priv/schema/rabbit.schema

Lines 44 to 46 in 4c57a42

	{mapping, "listeners.ssl", "rabbit.ssl_listeners",[
	{datatype, {enum, [none]}}
	]}.

For management.tcp there is something similar, but doesn't have the none value.

rabbitmq-server/deps/rabbitmq_management/priv/schema/rabbitmq_management.schema

Lines 47 to 48 in 4c57a42

	{mapping, "management.tcp.port", "rabbitmq_management.tcp_config.port",
	[{datatype, integer}]}.

Info

RabbitMQ: 3.13.7 (also briefly tested 4.0.2) via Docker image

[michaelklishin]

@warappa I do not see a single place in the docs that claims that what you are trying to do should be possible.

Multiple configuration files support was not added to allow for overriding of values set in "earlier" configuration files. They are supported to make it possible to incrementally generate the final rabbitmq.conf, namely specify generated credentials without messing with other user-provided values when RabbitMQ is deployed in a semi- or fully controlled environment.

The two files you pass are both passed to the library that parses, validates and translates .conf files. They are merged together in no particular order to form the final schema.

[michaelklishin]

As for a way to disable the regular TCP listener, it is as trivial as not configuring it at all.

Given the following rabbitmq.conf:

#
# Management
#

management.http_log_dir = /tmp/rabbitmq-http-api/logs/

management.ssl.port = 15671

management.ssl.cacertfile = /path/to/tls-gen.git/basic/result/ca_certificate.pem
management.ssl.certfile   = /path/to/tls-gen.git/basic/result/server_certificate.pem
management.ssl.keyfile    = /path/to/tls-gen.git/basic/result/server_key.pem

You will get the following listeners enabled:

rabbitmq-diagnostics listeners | grep "HTTP API"
# => Interface: [::], port: 15671, protocol: https, purpose: HTTP API over TLS (HTTPS)

In other words, there will be just one listener enabled. This is exactly what the HTTPS section of the docs claims, with joint HTTP/HTTPS setup being documented separately for this reason.

Is There Much Interest in an Alternative?

The management plugin already supports three different generations of settings and it is easily the most complex listeners-related part of the entire codebase.

You don't have to know much about Erlang or RabbitMQ to evaluate how complex this part is compared to its peers.

This is further complicated by the fact that there are other HTTP-based plugins such as STOMP and MQTT-over-WebSockets, the Prometheus plugin, and so on.

More changes to that area are very difficult to justify. They are not requested by paying users or regular contributors, in fact, we have seen requests of an exact opposite over the years,
which is why running two listeners is demonstrated in a dedicated doc section.

[warappa]

Thanks for your thorough reply.

I understand your point of historical complexity with the different configuration possibilities.
I was just surprised that the non-TLS messaging listeners can be overridden easily with none, but the (very popular) companion plugin "management" does not support that - therefore I submitted it initially as a bug as the configuration APIs do not align in this regard.

As this, as you expressed, will not be changed, maybe if the management plugin listener documentation would be adapted to be more explicit about it, it would help other developers with the similar assumtions.

(Referencing the other answer of yours)

They are merged together in no particular order to form the final schema.

Are the 2 files really parsed "in no particular order"? This seems to be contradicted by https://www.rabbitmq.com/docs/configure#config-confd-directory which states "...will be loaded in alphabetical order" - what is also my experience. But maybe I don't understand what you wanted to address.

Multiple configuration files support was not added to allow for overriding of values set in earlier configuration files.

Well, although not added for this purpose, you definitely can (to some degree) override values that were set previously. Which in my opinion is a widespread behavior and a good thing™.

[michaelklishin]

@warappa I will do a spike to see how straightforward it might end up being (specifically, whether this code path will be affected much or at all).

[michaelklishin]

I have a half-finished POC but for the next few weeks I will be swamped with something unrelated (build system and infrastructure changes). We'll see how it goes.

[warappa]

No problem! Thank you for looking into it!