Update transport version docs #133862

JVerwolf · 2025-08-29T21:04:41Z

Updates the docs for the new Transport Version model. This should only be merged once we are aready for this new model to be generally used.

github-actions · 2025-08-29T21:06:04Z

🔍 Preview links for changed docs

docs/internal/Versioning.md

elasticsearchmachine · 2025-08-29T21:06:45Z

Pinging @elastic/es-core-infra (Team:Core/Infra)

elasticsearchmachine · 2025-08-29T21:06:45Z

Pinging @elastic/core-docs (Team:Docs)

ldematte

I know it's probably early for a full review, as the section on details/description is still missing, but what is here so far looks clear and easy to follow to me.

prdoyle

I think this would benefit from a brief overview of the principles behind the TV allocation system, like what files there are, what they mean, and what the system invariants are. Then we can dive into the operational recipes.

prdoyle · 2025-09-09T20:23:41Z

docs/internal/Versioning.md

+args. Because this command is idempotent, it will detect that the transport version constant is no longer referenced,
+and remove the corresponding state file. E.g.:


I'd avoid this reference to idempotence here. Idempotence just means if you run it twice, it has the same effect as running it once, and that has no bearing on this behaviour. (I think you might be thinking more of "history independence" rather than idempotence here. They're related but not identical.)

As a user, I'd have the following question: suppose I merge a new TV, then in another PR I remove all references to that TV. Could that TV then be reused for someone else's PR?

prdoyle · 2025-09-09T20:25:55Z

docs/internal/Versioning.md

+To prevent race conditions, all transport version ids are first created on the `main` branch, after which they can be
+backported to prior release branches.


This term "race condition" is doing a lot of heavy lifting here. Perhaps something like this:

To prevent concurrently developed PRs from allocating the same transport version ID, ...

prdoyle · 2025-09-09T20:26:45Z

docs/internal/Versioning.md

-
-When in doubt, you can always leave the code as-is.
-This is an optional cleanup step that is never required for correctness.
+Backport state files can either be generated on the command line by gradle, or by adding a gihub version label to your


This is the first mention of "backport state files". We should probably define this term.

prdoyle · 2025-09-09T20:28:34Z

docs/internal/Versioning.md

+Backport state files can either be generated on the command line by gradle, or by adding a gihub version label to your
+PR.
+
+If you have already generated a transport version file, gradle will not be able to automatically detect which transport


If I'm not mistaken, this is describing a more unusual / exceptional situation. We should start by describing how a normal "canonical" backport works, and then this description will make more sense by describing the differences.

prdoyle · 2025-09-09T20:29:58Z

docs/internal/Versioning.md

+
+    ./gradlew :generateTransportVersion --name=my_new_tv --backport-branches=9.1,9.0
+
+If you later decide that you'd like to remove a backport, you can rerun the above command without the branch you want to


How much later? This stops working once the main PR is merged, right?

And all of this happens on the main branch, correct? If so, I think the part where we describe how to actually do backports to other branches is missing.

prdoyle · 2025-09-09T20:33:17Z

docs/internal/Versioning.md

+`transport/upper_bounds/` resource directory. To resolve the conflict, rerun the full generation command, and it will
+regenerate the state files, including the new transport version id. E.g.:
+
+    ./gradlew :generateTransportVersion --name=my_new_tv 


What happens to backports in this case?

docs/internal/Versioning.md

github-actions · 2025-09-09T23:10:35Z

ℹ️ Important: Docs version tagging

👋 Thanks for updating the docs! Just a friendly reminder that our docs are now cumulative. This means all 9.x versions are documented on the same page and published off of the main branch, instead of creating separate pages for each minor version.

We use applies_to tags to mark version-specific features and changes.

Expand for a quick overview

When to use applies_to tags:

✅ At the page level to indicate which products/deployments the content applies to (mandatory)
✅ When features change state (e.g. preview, ga) in a specific version
✅ When availability differs across deployments and environments

What NOT to do:

❌ Don't remove or replace information that applies to an older version
❌ Don't add new information that applies to a specific version without an applies_to tag
❌ Don't forget that applies_to tags can be used at the page, section, and inline level

🤔 Need help?

Check out the cumulative docs guidelines
Reach out in the #docs Slack channel

rjernst · 2025-09-09T23:15:30Z

@prdoyle I've just pushed a rework of most of these new docs. We want these to be developer focused, not get bogged down in details about how the new system works. For details we will heavily document the new system in TransportVersion.java. But here we want the docs to focus on how developers will interact with the new system.

Given the new changes I pushed, and the direction we are going for, can you take another look?

mosche · 2025-09-10T09:25:18Z

The most recent version of this looks really good, imho very intuitive and clear for someone who hasn't been involved in the heavy details of this. This is awesome and so much simpler!

ldematte

Two nits, but it is very easy to understand!
I think this is a great step ahead and it makes using and backporting TV very straightforward

docs/internal/Versioning.md

prdoyle

I'd recommend at least briefly describing how TVs are managed now: what the files are, what they contain, and what we do with them.

Having at least a broad mental model of what's going on can be really helpful to give devs an intuition for what command to run when.

prdoyle · 2025-09-11T16:04:59Z

docs/internal/Versioning.md

+You can also let CI handle updating transport versions for you. As version
+labels are updated on your PR, the generation task is automatically run with
+the appropriate backport branches and any changes to the internal state files
+are committed to your branch.


As a developer, my eyes would glaze over as soon as I see "let CI handle updating transport versions for you" and come away with the message that this is all automated and I don't need to keep reading.

If that's not the case, we should probably add a few words on the situations where this works and where it doesn't.

I think that's pretty much the takeaway we want. CI will "just work" in most cases. The only scenario in which you'd not just rely on CI to do it for you is if you had to verify these changes locally, in which case you'd have to apply the change manually, or let CI do it, and pull the automated change.

Ok that is great. In that case we might have buried the lede here a little. I wonder if we could call this out as a TL;DR somewhere prominent.

prdoyle · 2025-09-11T16:10:49Z

docs/internal/Versioning.md

+your transport version `my_tv` to `main` and `9.1`, and then realized you also
+needed to backport to `8.19` you would run (in `main`):
+
+    ./gradlew generateTransportVersion --name=my_tv --backport-branches=9.1,8.19


I see the value of making backport-branches the ultimate authority on backports; but also, in the specific user story you're describing here, that's counterproductive and error prone. (For example, suppose they failed to notice that 9.0 should also have been included.)

As a user, I'd like to say something like --add-backport-branches=8.19, or --backport-branches=+8.19, and know that it's not going to nuke any existing ones. (FWIW this is also idempotent.)

I think this is a fair point, but I would like to start without this, and add it in the future (it's a bit of complexity to the task implementation). The vast majority of the time developers can just let CI regenerate the transport version based on version labels.

Works for me!

docs/internal/Versioning.md

rjernst · 2025-09-11T18:05:48Z

@prdoyle I added a section on the layout of the resources files at a high level, hopefully this gives the overview you were hoping for (and it helped define "internal state files" which we use in several parts of this change).

prdoyle · 2025-09-11T20:06:28Z

docs/internal/Versioning.md

+### Internal state files
+
+The Elasticsearch server jar contains resource files representing each
+transport version. These files are loaded at runtime to construct
+`TransportVersion` instances. Since each transport version has its own file
+they can be backported without conflict.
+
+Additional resource files represent the latest transport version known on
+each release branch. If two transport versions are added at the same time,
+there will be a conflict in these internal state files, forcing one to be
+regenerated to resolve the conflict before merging to `main`.
+
+All of these internal state files are managed by gradle tasks; they should
+not be edited directly.


This is perfect. It's exactly what I had in mind.

Update TV docs

f576267

elasticsearchmachine added the v9.2.0 label Aug 29, 2025

JVerwolf marked this pull request as ready for review August 29, 2025 21:05

JVerwolf added >docs General docs changes :Core/Infra/Transport API Transport client API labels Aug 29, 2025

elasticsearchmachine added Team:Core/Infra Meta label for core/infra team Team:Docs Meta label for docs team labels Aug 29, 2025

rjernst self-assigned this Aug 29, 2025

ldematte reviewed Sep 1, 2025

View reviewed changes

prdoyle reviewed Sep 9, 2025

View reviewed changes

docs/internal/Versioning.md Outdated Show resolved Hide resolved

Update

4ef84a4

Merge branch 'main' into transport/update-docs

ee77afb

rjernst changed the title ~~[DNM] Update TV docs~~ Update transport version docs Sep 9, 2025

ldematte approved these changes Sep 10, 2025

View reviewed changes

docs/internal/Versioning.md Outdated Show resolved Hide resolved

docs/internal/Versioning.md Outdated Show resolved Hide resolved

feedback

04335d0

mark-vieira reviewed Sep 10, 2025

View reviewed changes

docs/internal/Versioning.md Show resolved Hide resolved

docs/internal/Versioning.md Show resolved Hide resolved

docs/internal/Versioning.md Show resolved Hide resolved

docs/internal/Versioning.md Show resolved Hide resolved

docs/internal/Versioning.md Outdated Show resolved Hide resolved

rjernst added 2 commits September 11, 2025 07:48

more feedback

29f2aa7

iter

bc7c9ac

prdoyle reviewed Sep 11, 2025

View reviewed changes

more feedback

482a031

prdoyle reviewed Sep 11, 2025

View reviewed changes

prdoyle approved these changes Sep 11, 2025

View reviewed changes

rjernst merged commit 948f661 into elastic:main Sep 11, 2025
11 checks passed

		args. Because this command is idempotent, it will detect that the transport version constant is no longer referenced,
		and remove the corresponding state file. E.g.:

		To prevent race conditions, all transport version ids are first created on the `main` branch, after which they can be
		backported to prior release branches.


		./gradlew :generateTransportVersion --name=my_new_tv --backport-branches=9.1,9.0

		If you later decide that you'd like to remove a backport, you can rerun the above command without the branch you want to

Update transport version docs #133862

Update transport version docs #133862

Uh oh!

Conversation

JVerwolf commented Aug 29, 2025

Uh oh!

github-actions bot commented Aug 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔍 Preview links for changed docs

Uh oh!

elasticsearchmachine commented Aug 29, 2025

Uh oh!

elasticsearchmachine commented Aug 29, 2025

Uh oh!

ldematte left a comment

Choose a reason for hiding this comment

Uh oh!

prdoyle left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

prdoyle Sep 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actions bot commented Sep 9, 2025

ℹ️ Important: Docs version tagging

When to use applies_to tags:

What NOT to do:

🤔 Need help?

Uh oh!

rjernst commented Sep 9, 2025

Uh oh!

mosche commented Sep 10, 2025

Uh oh!

ldematte left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

prdoyle left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

rjernst commented Sep 11, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actions bot commented Aug 29, 2025 •

edited

Loading

prdoyle left a comment •

edited

Loading

prdoyle Sep 9, 2025 •

edited

Loading