Using username in a TLS context is inappropriate

[dblas]

Is your feature request related to a problem? Please describe.

Authentication is mainly based upon username (at least for MQTT). Ok, fine.
In TLS context you don't need username since you have the certificate to authenticate the device.
Then, AFAIK, MQTT is made for connecting millions of objects.
How can you imagine managing millions of usernames?
And what for? To separate queues?
That's insane!
That's not at all the way it should work, above all in a TLS context where certificate is used AS the main authenticator.
In this case you'll have 3 databases to manage: the usernames' one, the clientIDs' one and the certificates' one (serial number) which is managed by the CA.
10k objects => 30k items to manage! Yes, that's insane.
In MQTT, at least, the id used for managing client is the ... clientID. That the ID used for managing client sessions.
And, as you may know, you can perfectly separate queues through ACLs using %c in order for one client to not be able to see other client's queues.

Describe the solution you'd like

In a TLS context, be able to extract the clientID from the certificate as it's possible to do it with username.
Thus,
CN (for example) => clientID
and SAN (email, DNS, whatever) => username (which is already feasible as of current beta).

This way, it's then possible to separate queues using the /%c/ trick.
No need to manage thousands of usernames, clientIDs are dynamically managed by the Server and we only have to manage serial numbers through the CA. Only one database and every one is happy.
Authentication can still occur against the username but this one can be unique for a pool of hundreds of devices.

Describe alternatives you've considered

Can't see other since managing thousands of usernames is insane if there is an alternative to separate queues.

Additional context

No response

[michaelklishin]

@dblas using highly emotional pitches will not work very well in this community. Instead of throwing around words like "insane" and asking questions that seem rhetorical to you but may or may be obvious to others, consider explaining what is it that you would like to see without judgement calls.

Or simply contribute a PR that would demonstrate it.

Throwing around words like "insane", "not at all how it should work" is great for letting steam out but not so much for improving open source software you very likely get entirely for free.

For x.509 certificate-based authentication scenarios, there is a built-in plugin that makes it possible to extract a username from the most common certificate fields.

On top of that, you don't have to use the internal database for users. LDAP or any HTTPS-based service with a tiny API allow you to manage bajillions of usernames however you please.

[michaelklishin]

@dblas I will be very blunt here: if this discussion continues like this, it will not get very far, and we won't let insults towards the core team slide.

Please take a few steps back and describe what is it that you're trying to do, why, what authN/authZ backends (and mechanisms) have you considered.

How MQTT client information is extracted from x.509 certificates is not a recent development. I don't think 3.13 changes anything. So somehow the current set of options available works for at least some users.

There's still about 7-8 weeks before 3.13.0 ships, and months before 4.0 does. Reasonable changes can be introduced if instead of spitting emotions and making bold claims ("That's not at all the way it should work") someone presents are specific set of changes without calling the core team insane.

[dblas]

I wrote an answer this morning but it doesn't seem to have gone through.
Whatever.

Hello,

I think there is a little misunderstanding there. I do not blame anyone nor I insult anyone. I'm simply raising an operational problem that, I suppose, takes its root in a misunderstanding about the identity management part of a MQ broker whatever it is: Mosquitto, RabbitMQ, VerneMQ, HiveMQ, EMXQ.

None of the brokers I had time to look at (RabbitMQ, VerneMQ, Mosquitto) are able to extract the clientid from a certificate.
I think there is a reason but which one?

At this moment I think that's because too much confidence was placed into the id/mdp working that prevails in many applications today. I understand, but as soon as you're using certificates, things change: authentication is then made against a CA and doesn't need to be done against an LDAP or equivalent. That'd be a double-use.

Doing both is a lack of time as well as resource-hungry. You can but that will be complicated to keep things synchronized.
When we talk about human beings bearing a certificate that's understandable to carry on with username since you'll have many applications against which the user will have to authenticate and which do not support certificate.

But in a restricted context as such (I)IoT is, that'll be far TOO complicated. We're talking about machines, automates, not humans beings.

Deeply sorry if I sounded incorrect. That wasn't intended. I always write as I think.
For your information, I made the same request for Mosquitto and will do it for VerneMQ.

db

[michaelklishin]

Now, to comment on some specific bits that can be deciphered from the original post:

CN (for example) => clientID

I am not convinced that a lot of MQTT tools do this by default. Client ID extraction instead of
using whatever the client provides could be a reasonable feature request but you make it sound
like the obvious solution that every other MQTT server has adopted. That's not the case. Client-provided IDs
are used very widely.

SAN (email, DNS, whatever) => username

Username extraction from Subject Alternative Name has been supported for years.

it's then possible to separate queues using the /%c/ trick

We'd need a clarification on what the well known "/c%/ trick" is. We do not guess in this community.

On the surface, it looks like the MQTT plugin is missing two things:

• A way to extract client ID from a client certificate field, and ignore the client-provided value
• Some "trick" that would make queue naming generation use a different value from what it does today?

Whether the plugin should use separate queues or not is an entirely separate discussion. Queues only offer destructive consumption, so either the plugin would need to support streams somehow (a change way out of scope for 3.13), or the use of separate queues with routing to several of them will be unavoidable in many cases. I don't see how this is relevant to how "insane" is what the plugin does w.r.t. username extraction.

[dblas]

Now, to comment on some specific bits that can be deciphered from the original post:

CN (for example) => clientID

I am not convinced that a lot of MQTT tools do this by default.

You are right. No MQTT brokers do this. Only some platforms (AWS, etc).
My request was a feature request for RabbitMQ not a statement or the result of a survey.

And I don't understand why it's as it. In MQTT it seems to me far more logical to use the clientID as an identifier than to use the username.
Maybe I've not the same mind logic - to avoid headaches - as most users have. Whatever ...

For me the ClientID is the true identifier and the session is based upon it. username can have its own use cases (to gather devices, to authenticate humans beings on the web interface, to create customer families) but clearly, if you have thousands of devices, you won't want to manage thousands of usernames whereas, if you are in a TLS context, you already manage thousands of certificates (serial numbers).

Client ID extraction instead of
using whatever the client provides could be a reasonable feature request but you make it sound
like the obvious solution that every other MQTT server does. That's not the case. Client-provided IDs
are used very widely.

SAN (email, DNS, whatever) => username

Username extraction from Subject Alternative Name has been supported for years.

it's then possible to separate queues using the /%c/ trick

We'd need a clarification on what the well known "/c%/ trick" is. We do not guess in this community.

%c and %u are the patterns used in VerneMQ for substituing respectively clientID (^{client_id}-.) and username (^{username}-.).

Sorry, I'm new to RabbitMQ, not accustomed to concepts yet. But, it's a product chosen by one of our subsidiaries and, as a group CISO, I have to specify a few rules to avoid wild usages.

On the surface, it looks like the MQTT plugin is missing two things:

A way to extract client ID from a client certificate field, and ignore the client-provided value

YEP that's it. That simple said like this.
Are we sure there won't be any other consequences on other flows (authentication, etc)?

Some "trick" that would make queue naming generation use a different value from what it does today?

If I understand right, these tricks already exist: ^{client_id}-.* and ^{username}-.* I didn't see them before my post. Sorry.

The idea is to use the client_id pattern into ACL rules.

Thus, writing that devices can only have access to /sthing1/sthing2/^{client_id}-.*/tpc1/tpc2/ means each devices is confined to its own queue in only one sentence. No need to write as many ACL rules as they are devices.

Whether the plugin should use separate queues or not is an entirely separate discussion. Queues only offer destructive consumption, so either the plugin would need to support streams somehow (a change way out of scope for 3.13), or the use of separate queues with routing to several of them will be unavoidable in many cases. I don't see how this is relevant to how "insane" is what the plugin does w.r.t. username extraction.

Username management insanity is related to operational constraint in a TLS context when you have thousands of devices: you'll have to manage them in LDAP, a database, whatever, whereas on another side you already manage devices serial numbers inside the AC that delivers the certificates.

Double-management, synch needed => headaches
Of course you could create many usernames in advance during the deployment phase but sincerely that's not clean at all and leads to this insanity landscape. Sorry for that.

I'm convinced that's a very good way to make RabbitMQ go along with the new MQTT engine that came with version 3.12.
But, reading you, I'm afraid we'll have to explain why to many current users.
db

[michaelklishin]

The only reason I can think is, client IDs are not used for much more than logging in a lot of MQTT installations. Plus to enforce "connection uniqueness" as per the MQTT spec.

We can add extraction of client ID from a certificate field if present. How do you suggest that it is used then? To override the client connection-provided value? Something else?

[michaelklishin]

Another thing that I'd need to test is whether we pass client ID to authN backends. If we do, and the extraction from the certificate is done, a tiny HTTP service you can develop would be able to authenticate clients based on their certificate-provided client IDs.

[dblas]

Having thinking about, yes, It seems relevant to push the clientID, once retrieved, into the same flows username follows (auth, authz). We don't know at this time what will be the usages.
For me it makes sense (of course) to use the clientID above all as an authenticator. As you said, its uniqueness is, in principle, garanteed which makes it a far better candidate for authentication and authorization workflows than username.
db

[ansd]

The MQTT spec differentiates between client ID and user name. The client ID must always be unique. The client ID is used to track the state of an MQTT session. In contrast, the user name is meant to be used for authentication and authorization.

I'm not 100% sure as to why the distinction between client ID and user name was made in the MQTT spec, but the most plausible explanation to me is that clients can re-use the same username for authentication and authorization while still having a unique client ID to differentiate the different sessions.

When the client authenticates with a certificate, couldn't you use the same user name in the SAN of the different certificates? This is already supported today by the RabbitMQ MQTT plugin and solves your problem to not maintain N different user names. Obviously, the clients will still connect with different client IDs in the CONNECT packet.

In TLS context you don't need username since you have the certificate to authenticate the device.

Authentication is not enough. You still need a username for RabbitMQ to perform authorization, right?

Extracting a client ID from the certificate doesn't make sense to me. The client ID is mandatory (albeit it could be empty) and already sent in the CONNECT packet from client to server.
I'd prefer to keep the client ID as a means to track session state, and the user name as a means to perform authentication and authorization.

[michaelklishin]

If you use the HTTP authN/authZ backend that was recommended several times above, you can ignore the username entirely in your service's implementation (and return a constant value that will be displayed everywhere). Not having a user on a connection is not an option in RabbitMQ, by design. Even MQTT's anonymous mode is backed by a user (the default user configurable for the plugin).

[michaelklishin]

Using client ID instead of username in queue names can also be an option if it does not affect the publishing or delivery path (that is, if we need to generate the name once when a client subscribes).

Introducing such optionality on the hot code paths will make the plugin less efficient for everyone, including those who do not have any of the needs expressed in this thread.

This increasingly sounds like a request to re-purpose client ID for username. I am not convinced that's how most MQTT implementations approach access control. The spec certainly does not suggest that client IDs can entirely replace usernames.

[dblas]

I don't think so since who talks about username will need to manage it (both locations as I explained).
Please consider the clientId as we discuss and let the users choose if they want to use the username (and manage it into the AC as well as into something else) => AuthN or if they want to use the clientID (no need to manage except into the AC) => AuthN.
db

[michaelklishin]

I'm not sure I understand what specifically you are asking for @dblas. "Use" for what specifically?

[dblas]

Use?
Did you read the whole thread, discussion with David in particular?
I was thinking my objectives were clear why I privilege the use of clientID rather than the username.
I explain why it's inefficient and a waste of time to use username when you have thousands of devices which have, of course, already a unique id by themselves: materialized by the clientID.
That's valid for MQTT and could be different in AMPQ (didn't look so far).
db

[michaelklishin]

#9323 and #9322 are two outcomes of this discussion.

#9322 is a no-brainer. #9323 does not make much sense given that

• Client ID is not a means of authentication, although it potentially can be used as an extra factor in authN decision making
• In 3.12, client ID is already used in queue names together with QoS level
• With an external HTTP-backed service, assuming that MQTT: make sure that client ID is propagated to authN backends #9322 is done, the user will have a lot of flexibility when it comes to authN decisions

but we may add it anyway. There can be some value in encoding as much information as possible into the certificate.