Skip to content

Conversation

valeriy42
Copy link
Contributor

@valeriy42 valeriy42 commented Sep 11, 2025

Since the 9.0 upgrade, we have introduced a solution by setting large .ml-anomalies to read only during the upgrade assistant. However, there is an edge case that will cause the obsolete model snapshot to fail to upgrade.

Step to reproduce:

  1. setup a 7.17 cluster
  2. create and start an anomaly detection job
  3. upgrade to 8.18 or 8.19
  4. go to upgrade assistant
  5. change .ml-anomalies-shared to read only
  6. then try upgrade obsolete model snapshot

Observed:

  • upgrade model snapshot will fail.

This PR extends the warning message explaining the steps to resolve the issue.

@elasticsearchmachine elasticsearchmachine added needs:triage Requires assignment of a team area label v9.2.0 labels Sep 11, 2025
@valeriy42 valeriy42 added v9.1.4 v8.19.4 v8.18.8 v9.0.8 :ml Machine learning Team:ML Meta label for the ML team >non-issue and removed needs:triage Requires assignment of a team area label labels Sep 11, 2025
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/ml-core (Team:ML)

@valeriy42 valeriy42 requested review from DaveCTurner and davidkyle and removed request for DaveCTurner September 11, 2025 12:40
Copy link
Contributor

@DaveCTurner DaveCTurner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know you picked the wrong David but I'm replying anyway 😁

Copy link
Contributor

@DaveCTurner DaveCTurner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@valeriy42 valeriy42 changed the title extend log message [ML] Improve log message for model snapshot update Sep 12, 2025
@valeriy42 valeriy42 added the auto-backport Automatically create backport pull requests when merged label Sep 12, 2025
@valeriy42 valeriy42 merged commit c47522c into elastic:main Sep 15, 2025
35 checks passed
@elasticsearchmachine
Copy link
Collaborator

💔 Backport failed

Status Branch Result
9.1 Commit could not be cherrypicked due to conflicts
8.19 Commit could not be cherrypicked due to conflicts
8.18 Commit could not be cherrypicked due to conflicts
9.0 Commit could not be cherrypicked due to conflicts

You can use sqren/backport to manually backport by running backport --upstream elastic/elasticsearch --pr 134541

mridula-s109 pushed a commit to mridula-s109/elasticsearch that referenced this pull request Sep 17, 2025
Since the 9.0 upgrade, we have introduced a solution by setting large .ml-anomalies to read only during the upgrade assistant. However, there is an edge case that will cause the obsolete model snapshot to fail to upgrade.

Step to reproduce:

setup a 7.17 cluster
create and start an anomaly detection job
upgrade to 8.18 or 8.19
go to upgrade assistant
change .ml-anomalies-shared to read only
then try upgrade obsolete model snapshot
Observed:

upgrade model snapshot will fail.
This PR extends the warning message explaining the steps to resolve the issue.
valeriy42 added a commit to valeriy42/elasticsearch that referenced this pull request Sep 17, 2025
Since the 9.0 upgrade, we have introduced a solution by setting large .ml-anomalies to read only during the upgrade assistant. However, there is an edge case that will cause the obsolete model snapshot to fail to upgrade.

Step to reproduce:

setup a 7.17 cluster
create and start an anomaly detection job
upgrade to 8.18 or 8.19
go to upgrade assistant
change .ml-anomalies-shared to read only
then try upgrade obsolete model snapshot
Observed:

upgrade model snapshot will fail.
This PR extends the warning message explaining the steps to resolve the issue.

(cherry picked from commit c47522c)
valeriy42 added a commit to valeriy42/elasticsearch that referenced this pull request Sep 17, 2025
Since the 9.0 upgrade, we have introduced a solution by setting large .ml-anomalies to read only during the upgrade assistant. However, there is an edge case that will cause the obsolete model snapshot to fail to upgrade.

Step to reproduce:

setup a 7.17 cluster
create and start an anomaly detection job
upgrade to 8.18 or 8.19
go to upgrade assistant
change .ml-anomalies-shared to read only
then try upgrade obsolete model snapshot
Observed:

upgrade model snapshot will fail.
This PR extends the warning message explaining the steps to resolve the issue.

(cherry picked from commit c47522c)

# Conflicts:
#	server/src/main/java/org/elasticsearch/common/ReferenceDocs.java
#	server/src/main/resources/org/elasticsearch/common/reference-docs-links.txt
@valeriy42
Copy link
Contributor Author

💚 All backports created successfully

Status Branch Result
9.1
9.0
8.19
8.18

Questions ?

Please refer to the Backport tool documentation

valeriy42 added a commit to valeriy42/elasticsearch that referenced this pull request Sep 17, 2025
Since the 9.0 upgrade, we have introduced a solution by setting large .ml-anomalies to read only during the upgrade assistant. However, there is an edge case that will cause the obsolete model snapshot to fail to upgrade.

Step to reproduce:

setup a 7.17 cluster
create and start an anomaly detection job
upgrade to 8.18 or 8.19
go to upgrade assistant
change .ml-anomalies-shared to read only
then try upgrade obsolete model snapshot
Observed:

upgrade model snapshot will fail.
This PR extends the warning message explaining the steps to resolve the issue.

(cherry picked from commit c47522c)

# Conflicts:
#	server/src/main/java/org/elasticsearch/common/ReferenceDocs.java
#	server/src/main/resources/org/elasticsearch/common/reference-docs-links.txt
valeriy42 added a commit to valeriy42/elasticsearch that referenced this pull request Sep 17, 2025
Since the 9.0 upgrade, we have introduced a solution by setting large .ml-anomalies to read only during the upgrade assistant. However, there is an edge case that will cause the obsolete model snapshot to fail to upgrade.

Step to reproduce:

setup a 7.17 cluster
create and start an anomaly detection job
upgrade to 8.18 or 8.19
go to upgrade assistant
change .ml-anomalies-shared to read only
then try upgrade obsolete model snapshot
Observed:

upgrade model snapshot will fail.
This PR extends the warning message explaining the steps to resolve the issue.

(cherry picked from commit c47522c)

# Conflicts:
#	server/src/main/java/org/elasticsearch/common/ReferenceDocs.java
#	server/src/main/resources/org/elasticsearch/common/reference-docs-links.txt
elasticsearchmachine pushed a commit that referenced this pull request Sep 17, 2025
Since the 9.0 upgrade, we have introduced a solution by setting large .ml-anomalies to read only during the upgrade assistant. However, there is an edge case that will cause the obsolete model snapshot to fail to upgrade.

Step to reproduce:

setup a 7.17 cluster
create and start an anomaly detection job
upgrade to 8.18 or 8.19
go to upgrade assistant
change .ml-anomalies-shared to read only
then try upgrade obsolete model snapshot
Observed:

upgrade model snapshot will fail.
This PR extends the warning message explaining the steps to resolve the issue.

(cherry picked from commit c47522c)

# Conflicts:
#	server/src/main/java/org/elasticsearch/common/ReferenceDocs.java
#	server/src/main/resources/org/elasticsearch/common/reference-docs-links.txt
elasticsearchmachine pushed a commit that referenced this pull request Sep 17, 2025
Since the 9.0 upgrade, we have introduced a solution by setting large .ml-anomalies to read only during the upgrade assistant. However, there is an edge case that will cause the obsolete model snapshot to fail to upgrade.

Step to reproduce:

setup a 7.17 cluster
create and start an anomaly detection job
upgrade to 8.18 or 8.19
go to upgrade assistant
change .ml-anomalies-shared to read only
then try upgrade obsolete model snapshot
Observed:

upgrade model snapshot will fail.
This PR extends the warning message explaining the steps to resolve the issue.

(cherry picked from commit c47522c)

# Conflicts:
#	server/src/main/java/org/elasticsearch/common/ReferenceDocs.java
#	server/src/main/resources/org/elasticsearch/common/reference-docs-links.txt
elasticsearchmachine pushed a commit that referenced this pull request Sep 17, 2025
Since the 9.0 upgrade, we have introduced a solution by setting large .ml-anomalies to read only during the upgrade assistant. However, there is an edge case that will cause the obsolete model snapshot to fail to upgrade.

Step to reproduce:

setup a 7.17 cluster
create and start an anomaly detection job
upgrade to 8.18 or 8.19
go to upgrade assistant
change .ml-anomalies-shared to read only
then try upgrade obsolete model snapshot
Observed:

upgrade model snapshot will fail.
This PR extends the warning message explaining the steps to resolve the issue.

(cherry picked from commit c47522c)

# Conflicts:
#	server/src/main/java/org/elasticsearch/common/ReferenceDocs.java
#	server/src/main/resources/org/elasticsearch/common/reference-docs-links.txt
elasticsearchmachine pushed a commit that referenced this pull request Sep 17, 2025
Since the 9.0 upgrade, we have introduced a solution by setting large .ml-anomalies to read only during the upgrade assistant. However, there is an edge case that will cause the obsolete model snapshot to fail to upgrade.

Step to reproduce:

setup a 7.17 cluster
create and start an anomaly detection job
upgrade to 8.18 or 8.19
go to upgrade assistant
change .ml-anomalies-shared to read only
then try upgrade obsolete model snapshot
Observed:

upgrade model snapshot will fail.
This PR extends the warning message explaining the steps to resolve the issue.

(cherry picked from commit c47522c)
gmjehovich pushed a commit to gmjehovich/elasticsearch that referenced this pull request Sep 18, 2025
Since the 9.0 upgrade, we have introduced a solution by setting large .ml-anomalies to read only during the upgrade assistant. However, there is an edge case that will cause the obsolete model snapshot to fail to upgrade.

Step to reproduce:

setup a 7.17 cluster
create and start an anomaly detection job
upgrade to 8.18 or 8.19
go to upgrade assistant
change .ml-anomalies-shared to read only
then try upgrade obsolete model snapshot
Observed:

upgrade model snapshot will fail.
This PR extends the warning message explaining the steps to resolve the issue.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
auto-backport Automatically create backport pull requests when merged backport pending :ml Machine learning >non-issue Team:ML Meta label for the ML team v8.18.8 v8.19.4 v9.0.8 v9.1.4 v9.2.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants