Upgrade from 7.17 guide

[eedugon]

This PR adds a new ad-hoc document to guide users to upgrade from 7.17.x to 9.x, which requires two major upgrades plus upgrading all ingest components and clients between the upgrades.

It supports all deployment types, covers orchestrator upgrades, etc.

This new document builds upon and links to the existing upgrade documentation.

Its purpose is to bring together all relevant upgrade steps and considerations into a single, end-to-end guide that walks through what would normally be an exercise each user performs independently when planning a full stack upgrade, considering the specifics of this upgrade path.

It is not intended to introduce new upgrade procedures or instructions, as these should already be accurately documented in the existing upgrade guides.

Closes https://github.com/elastic/docs-content-internal/issues/387

cc: @ppf2 / @shainaraskas

[github-actions]

🔍 Preview links for changed docs

• deploy-manage/upgrade.md
• deploy-manage/upgrade/deployment-or-cluster.md
• deploy-manage/upgrade/deployment-or-cluster/upgrade-717.md
• deploy-manage/upgrade/prepare-to-upgrade.md

[shainaraskas]

placeholder added: #3409

dropped eol from the URL and added to the toc as hidden

[kunisen]

Hi @eedugon thanks for the PR! This is very helpful!

I have one thing to ask for your insights:
Could we add a note that if user use API way to upgrade on ECH, they need to run GET /_migration related APIs to get their clusters ready to upgrade please?
　
This API set is called by Kibana upgrade assistant, but if user use terraform or other ways to upgrade from cloud, they can bypass upgrade assistant, and cause troubles.
https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-migration
　
FWIW, we have an internal ticket for this.

cc @Gunnerva as this is under your area.

[eedugon]

@kunisen , thanks a lot for sharing that interesting issue.

We'll definitely add that information, but it needs to be added in the standard docs also, as that information is relevant in general for upgrades on ECH using the API (i.e when using terraform as you said). I'll find the best way to address it.

Then in this 7.17->9.x ad-hoc guide we will also highlight it.

cc: @shainaraskas , probably the issue that @kunisen has addressed deserves a separate issue, WDYT?

[shainaraskas]

@eedugon yes it can be handled separately

[shainaraskas]

I think you're on the right track. I mostly made cosmetic edits.

for ingest, I think we need some more targeted links. otherwise I think we're striking the right balance

we can always build on this if people are reporting specific difficulties.

[ppf2]

There are two places in the guide that say:

review the complete list of breaking changes and known issues during your preparation phase.

How about the following to be clear that they need to click on every in between version's migration guide 😄

review the complete list of breaking changes and known issues in every release from your current version through your target version during your preparation phase.

[ppf2]

Typically, libraries are upgraded to match the Elasticsearch version after each of the upgrades.

On this note, it seems like there can be a client "outage" for Java clients between the time the Elasticsearch version is upgraded to the next major version and when they upgrade their client libraries to match. The linked doc states that the java client will only work with same or higher minor version without breaking. So from a planning point of view, they may need to have 2 client applications running different library versions in parallel or need the ability to quickly hot swap client library versions as soon as the server version is upgraded.

[ppf2]

In the https://www.elastic.co/docs/deploy-manage/upgrade/prepare-to-upgrade#run-the-upgrade-assistant which is cross-referenced from this guide, it says:

Indices created in 7.x or earlier must be reindexed, deleted, or archived (marked as read-only) before upgrading to 9.x.

The archived link takes you to a page that only talks about indices created in versions 5 or 6. So what about indices created in 7.x? Are they not supported by the mark-as-read-only/archive feature? Can we confirm with the devs and update the docs accordingly? Thx!

[eedugon]

Thanks @ppf2 !! I'll keep track of all your comments, let me share something to begin with:

breaking changes comment
change applied, good point!

archived functionality

I'm 99% sure that's a docs bug caused during the docs migration, and we didn't adapt the doc to the 9.x reality (it says indices on v5 and v6 because that content was created in 8.x, where 7.x indices didn't need archive).

Indices in 7.x are definitely valid for archive functionality, and I've tested it when testing the uggrade from 7.17 to 9.x. I'll create an issue to address the bug in a separate issue.

client libraries and possible outage because it appears they are not compatible with next major

For the clients issue I really hope and think that all our clients in 7.17 are compatible with 8.19 generally speaking, of course they could be affected by breaking changes but in general they should work. And also the REST API Compatibility mode should be an option for users to consider. I had already updated the docs on this regard giving better visibility to this (I agree the docs could make the impression of an outage).

We have this doc and worklow created since 8.0 docs, and I really hope all client libraries are aligned on that regard.

My plan to address this is:

• Confirm with devs and docs owners of the clients what's the expectation of the clients for major upgrades, as the client docs are not clear at all (except the python doc which is aligned the behavior I mentioned).

• In the docs I think we should highlight that use cases with custom developed applications are an ideal case for considering testing the entire procedure in a non-prod environment, which is something we already recommend but it might be worthy to mention it on relation with client libraries (cc: @georgewallace :-D ).

[shainaraskas]

I'm almost done with my review but going to a meeting. can't say enough good things about what I have read so far - this guide is really complex but super clean and easy to follow, and just packed with great info. will make people feel confident in the upgrade process 🚀

going to provisionally approve because nothing so far is blocking, just in case you need to ship timewise

[shainaraskas]

we should also surface this doc on the parent page

here: https://www.elastic.co/docs/deploy-manage/upgrade/deployment-or-cluster

and here: https://www.elastic.co/docs/deploy-manage/upgrade

we should also consider surfacing 7.x upgrade paths here (or at least linking to the source of truth): https://www.elastic.co/docs/deploy-manage/upgrade#upgrade-paths

[eedugon]

done

[shainaraskas]

this looks good, but be careful about assuming that the design of this note will always remain the same. honestly I'm cool with shipping this, but this component could be redesigned and then your sentence will be broken. something to keep in mind as you use these components in future :) would go with plain admonition blocks if you want custom title text

[eedugon]

done, converted to admonition

[shainaraskas]

just a couple more comments (done reviewing). looks great.