Add note about transport mTLS

[DaveCTurner]

Until recently, public CAs would issue certificates with an Extended Key
Usage set that includes the clientAuth value, allowing these
certificates to be used for mTLS. Nonetheless it is a mistake to use
such certificates for mTLS. To prevent users from continuing to make
this mistake, all certificates issued by public CAs will soon omit the
clientAuth usage value.

Elasticsearch will by default use mTLS for inter-node connections, and
we've recently encountered some users who have been obtaining their
transport certificates from such public CAs and mistakenly using them
for mTLS. This commit adds some documentation clarifying the security
model and giving clearer recommendations in this area.

[elasticsearchmachine]

Pinging @elastic/es-security (Team:Security)

[elasticsearchmachine]

Pinging @elastic/core-docs (Team:Docs)

[github-actions]

🔍 Preview links for changed docs

• docs/reference/elasticsearch/configuration-reference/security-settings.md

[github-actions]

ℹ️ Important: Docs version tagging

👋 Thanks for updating the docs! Just a friendly reminder that our docs are now cumulative. This means all 9.x versions are documented on the same page and published off of the main branch, instead of creating separate pages for each minor version.

We use applies_to tags to mark version-specific features and changes.

Expand for a quick overview

When to use applies_to tags:

✅ At the page level to indicate which products/deployments the content applies to (mandatory)
✅ When features change state (e.g. preview, ga) in a specific version
✅ When availability differs across deployments and environments

What NOT to do:

❌ Don't remove or replace information that applies to an older version
❌ Don't add new information that applies to a specific version without an applies_to tag
❌ Don't forget that applies_to tags can be used at the page, section, and inline level

🤔 Need help?

• Check out the cumulative docs guidelines
• Reach out in the #docs Slack channel

[tvernum]

I'd like to strengthen this a little, though I don't have suggested wording right now.

I fear, as it is, people will turn on client authentication "because it solves their problem", (much like they set verification mode to node) without fully understanding the importance of that leading If.

Perhaps some warning about how serious a problem it can be if a rogue node joins your cluster, and that mTLS is the technique ES uses to prevent that, would be sufficient.

[DaveCTurner]

++ makes sense, I made it more opinionated

[shainaraskas]

this is a lot of dense best practice stuff. consider moving most of this guidance info into the instructional docs, and just hint here that there are some security best practices to follow (and link out)

possible locations (didn't look super deeply, but I can if desired)

https://www.elastic.co/docs/deploy-manage/security/secure-cluster-communications
https://www.elastic.co/docs/deploy-manage/security/set-up-basic-security
https://www.elastic.co/docs/deploy-manage/security/self-tls (or a new child page here)

[DaveCTurner]

Thanks @shainaraskas - I agree that this would be better-placed in the reference docs and would support moving it there in due course. However, right now the general reference docs only discuss obtaining certificates using elasticsearch-certutil which does the Right Thing™ in this area automatically. We don't yet have any docs talking about how to manage your TLS certificates using an external CA, and writing those docs would be quite some work. Yet, there's a big change in the CA industry happening right now (1 2 3 4 5 ...) that is causing problems for customers (mis-)using external CAs which we need to address in the docs as a matter of some urgency.

[shainaraskas]

wont' block this from being merged, but I still think there's room in our other docs for this info even if our current content doesn't have E2E procedures. my biggest concern here is visibility/discoverability because this information is urgent / causing problems.

I'd consider creating an H2 above the generate step on this page: https://www.elastic.co/docs/deploy-manage/security/set-up-basic-security titled something like Considerations for using an external CA (I believe this page is the entrypoint for setting up external certs).

if you would prefer to ship this here, we should consider adding a warning on that page (and maybe some of the other TLS encryption pages) that links to this, so people can more easily discover it. let me know what you prefer.

wonder if this info is also relevant to ECK transport certs.

[DaveCTurner]

I'd consider creating an H2 above the generate step on this page

I see, yes, makes sense. Could I ask you to do that @shainaraskas instead of this change? If you don't have capacity either then we can live with this PR for now but if you do then I would prefer it to be better-integrated onto that page.

[DaveCTurner]

wonder if this info is also relevant to ECK transport certs.

Yes, it is germane in the Issue node transport certificates with third-party tools section

[tvernum]

Thanks!

[shainaraskas]

@DaveCTurner working on this now

[shainaraskas]

@DaveCTurner I've created a PR to move the bulk of the recommendations to the instructional docs, and also add lots of warnings in any spot where people might be assessing using an external CA. as a result, I think we can whittle this down to the following. the linked PR will need to ship first but this can follow it shortly after.

Suggested change

	By default {{es}} uses mutual TLS (mTLS) to ensure the security of node-to-node transport connections within a cluster. Mutual TLS means that data is encrypted in transit, ensuring confidentiality and integrity, and also that both nodes in a connection must present a valid certificate to the other node when establishing the connection. Each {{es}} node requires that the certificate presented by the other node is issued by a certificate authority that it trusts for this purpose, ensuring that the nodes are authorized to establish this connection. The set of certificate authorities that a node trusts to issue certificates for transport connections is defined with settings in the `xpack.security.transport.ssl.*` namespace such as `xpack.security.transport.ssl.certificate_authorities` and `xpack.security.transport.ssl.truststore.path`. Certificates used for mTLS either must have no Extended Key Usage extension, or must have an Extended Key Usage extension that includes the `clientAuth` and `serverAuth` values.
	When using mTLS you must obtain your transport certificates from a certificate authority that only issues certificates to {{es}} nodes which are permitted to connect to your cluster. Do not use a public certificate authority, nor an organization-wide private certificate authority, because such certificate authorities issue certificates to entities other than the {{es}} nodes which are permitted to connect to your cluster. Public certificate authorities generally issue certificates with an Extended Key Usage extension that omits the `clientAuth` value and therefore cannot be used for mTLS anyway. The recommended best practice is to use a different private certificate authority for each {{es}} cluster, and not to use these certificate authorities for any other purpose.
	::::{warning}
	Anyone who can obtain a certificate from a certificate authority that your {{es}} cluster trusts for mTLS on transport connections will be able to use this certificate to establish a transport connection with a node in your cluster. A malicious actor with such a certificate may be able to use such a transport connection to invoke certain system-internal APIs, some of which may allow them to read or modify the data in your cluster.
	::::
	The security requirements for transport certificates (as defined by the `xpack.security.transport.ssl.*` settings) are significantly different from the security requirements for HTTP certificates (as defined by the `xpack.security.http.ssl.*` settings). HTTP connections do not generally use mTLS since HTTP has its own authentication mechanisms, so HTTP certificates do not usually need to include the `clientAuth` value in their Extended Key Usage extension. It often makes sense to obtain the nodes' HTTP certificates from a public certificate authority, or from an organization-wide private certificate authority. It is almost always a mistake to use the same certificate for both HTTP and transport connections.
	If your environment has some other way to prevent unauthorized node-to-node connections, you may prefer not to use mTLS for transport connections. In this case, turn off mTLS for transport connections by setting `xpack.security.transport.ssl.client_authentication: none`. You may still use (non-mutual) TLS to ensure the confidentiality and integrity of node-to-node traffic by setting `xpack.security.transport.ssl.enabled: true`. If you are using non-mutual TLS for transport connections then your transport certificates do not require an Extended Key Usage extension which includes the `clientAuth` value.
	::::{warning}
	If you turn off mTLS by setting `xpack.security.transport.ssl.client_authentication` to `optional` or `none` then anyone with network access to a node in your {{es}} cluster will be able to establish a transport connection with that node. A malicious actor with this access may be able to use such a transport connection to invoke certain system-internal APIs, some of which may allow them to read or modify the data in your cluster. Use mTLS to protect your node-to-node transport connections unless you are absolutely certain that unauthorized network access to these nodes cannot occur.
	::::
	By default, {{es}} uses mutual TLS (mTLS) to secure node-to-node transport connections within a cluster. With mTLS, data is encrypted in transit and both nodes must present valid certificates when connecting. Each node requires that certificates be issued by a trusted certificate authority, ensuring that only authorized nodes can connect. [Learn about configuring node-to-node mTLS](docs-content://deploy-manage/security/secure-cluster-communications.md#encrypt-internode-communication).
	:::{warning}
	Transport connections between {{es}} nodes are security-critical and you must protect them carefully. Malicious actors who can observe or interfere with node-to-node transport traffic can read or modify cluster data. A malicious actor who can establish a transport connection might be able to invoke system-internal APIs, including APIs that read or modify cluster data.
	If you choose to issue node transport certificates using an external certificate authority, then carefully review [Using an external certificate authority to secure node-to-node connections](docs-content://deploy-manage/security/self-tls-considerations.md) to ensure that the certificates that you provide meet the security requirements.
	If your environment has some other way to prevent unauthorized node-to-node connections, you might prefer not to use mTLS for transport connections. [Learn more](docs-content://deploy-manage/security/external-ca-transport.md#turn-off-mtls).
	:::

[DaveCTurner]

++ thanks, I'll apply this when elastic/docs-content#3932 is merged

[eedugon]

Great improvements in this PR, but in one area I think it adds the wrong impression that for mTLS (in general) the server certs need both clientAuth and serverAuth, when that only applies to the specifics of Elasticsearch transport, where the same cert is used by the node acting as client and a server.

I've suggested a few small changes that should clarify this a bit better (hopefully) without changing the essence.

The proposals are for:

• Ensure we specify {{es}} transport when we say something that could be felt like a server cert without clientAuth extension is not suitable for mTLS.
• Explain why Elasticsearch transport cert requires both extension values or omit the extension (added a because short clause).
• Rephrase a bit the HTTP related paragrapth to remove the relation with mTLS (public CA-signed certs could be used for Elasticsearch HTTP server even if mTLS is required by the user).

[eedugon]

IMO the following clause is misleading (the part after therefore).

Public certificate authorities generally issue certificates with an Extended Key Usage extension that omits the clientAuth value and therefore cannot be used for mTLS anyway

It cannot be used for this Elasticsearch specific use case of mTLS, not mTLS in general (cannot be used for mTLS anyway feels generic).

Generally speaking, a public cert without clientAuth on the "server side" can be used for mTLS without issues, as clients and servers are expected to use their own certificates.

Let me know your thoughts about this suggestion.

Suggested change

When using mTLS you must obtain your transport certificates from a certificate authority that only issues certificates to {{es}} nodes which are permitted to connect to your cluster. Do not use a public certificate authority, nor an organization-wide private certificate authority, because such certificate authorities issue certificates to entities other than the {{es}} nodes which are permitted to connect to your cluster. Public certificate authorities generally issue certificates with an Extended Key Usage extension that omits the `clientAuth` value and therefore cannot be used for mTLS anyway. The recommended best practice is to use a different private certificate authority for each {{es}} cluster, and not to use these certificate authorities for any other purpose.

When using mTLS you must obtain your transport certificates from a certificate authority that only issues certificates to {{es}} nodes which are permitted to connect to your cluster. Do not use a public certificate authority, nor an organization-wide private certificate authority, because such certificate authorities issue certificates to entities other than the {{es}} nodes which are permitted to connect to your cluster. Public certificate authorities generally issue certificates with an Extended Key Usage extension that omits the `clientAuth` value and therefore cannot be used for {{es}} transport mTLS. The recommended best practice is to use a different private certificate authority for each {{es}} cluster, and not to use these certificate authorities for any other purpose.

This suggestion together with the previous clarification about the reasoning of needing both clientAuth and serverAuth should improve the flow.

[DaveCTurner]

Technically correct but I think the distinction is harmful to understanding in this context. If you know TLS well enough to understand this then you probably don't need to read these words, and if you don't then it may end up causing confusion.

Public CAs are all moving to remove the clientAuth EKU already and many of them describe this in their blog posts as removing support for mTLS. Practically speaking mTLS means the same thing as client-certificate authentication - if you take that away, you just have plain TLS.

[eedugon]

Suggested change

By default {{es}} uses mutual TLS (mTLS) to ensure the security of node-to-node transport connections within a cluster. Mutual TLS means that data is encrypted in transit, ensuring confidentiality and integrity, and also that both nodes in a connection must present a valid certificate to the other node when establishing the connection. Each {{es}} node requires that the certificate presented by the other node is issued by a certificate authority that it trusts for this purpose, ensuring that the nodes are authorized to establish this connection. The set of certificate authorities that a node trusts to issue certificates for transport connections is defined with settings in the `xpack.security.transport.ssl.*` namespace such as `xpack.security.transport.ssl.certificate_authorities` and `xpack.security.transport.ssl.truststore.path`. Certificates used for mTLS either must have no Extended Key Usage extension, or must have an Extended Key Usage extension that includes the `clientAuth` and `serverAuth` values.

By default {{es}} uses mutual TLS (mTLS) to ensure the security of node-to-node transport connections within a cluster. Mutual TLS means that data is encrypted in transit, ensuring confidentiality and integrity, and also that both nodes in a connection must present a valid certificate to the other node when establishing the connection. Each {{es}} node requires that the certificate presented by the other node is issued by a certificate authority that it trusts for this purpose, ensuring that the nodes are authorized to establish this connection. The set of certificate authorities that a node trusts to issue certificates for transport connections is defined with settings in the `xpack.security.transport.ssl.*` namespace such as `xpack.security.transport.ssl.certificate_authorities` and `xpack.security.transport.ssl.truststore.path`. Certificates used for {{es}} transport mTLS must either have no Extended Key Usage extension, or include both the `clientAuth` and `serverAuth` values, because the same certificate is used by the node when acting as client and server.

I think this clarification is critical to understand other parts of the doc.
Otherwise we give the wrong impression that certificates without clientAuth are not suitable for mTLS in general while they are perfectly valid for 99% of mTLS cases where clients and servers are different entities and use their own certs.

[DaveCTurner]

@eedugon this content has substantially moved over to elastic/docs-content#3932 and we're going to re-work it down to link to the new content shortly.

This change is driven by a need to explain the ongoing change in the CA industry to remove the clientAuth EKU from issued certificates, motivated by the fact that these certificates should only be used for server authentication and not client authentication (which the blog posts all seem to call mTLS). It's particularly targetted at folks that do not understand TLS properly and have been raising support cases with us about this change. So although what you're saying is technically correct if you distinguish the two certificates involved in a mTLS connection, in practice I think the distinction is going to be lost on the target audience and will cause more confusion than it solves. For the purposes of these docs, consider "mTLS" to be synonymous "client certificate authentication".

If it turns out that more clarity is needed in this area then we can refine it later, but I think in that case it'll need more substantial rework than replacing "mTLS" with "Elasticsearch transport mTLS" as you've suggested.

[eedugon]

If it turns out that more clarity is needed in this area then we can refine it later, but I think in that case it'll need more substantial rework than replacing "mTLS" with "Elasticsearch transport mTLS" as you've suggested.

Well, the suggestion is because implying that a server cert without clientAuth is not valid for mTLS could be misleading, where the fact is that it's not valid for us, for the Elasticsearch transport, and because the entity is a client and a server at the same time.

I'm trying to 1st help, and 2nd trying to provide the smallest changes possible to make the content accurate. But anyway, whatever you feel it's better. We could refine at a later stage, for sure.