Skip to content

[BUG] illegal_argument_exception for snapshot policyΒ #1604

@Jakob3xD

Description

@Jakob3xD

What is the bug?
I have an opensearch cluster that repetitively fails to execute policies for indices that contain an alias action.
It also causes the explain endpoint to fail GET _plugins/_ism/explain/*

{
  "error": {
    "root_cause": [
      {
        "type": "illegal_argument_exception",
        "reason": "Alias action can only work on its applied index so don't accept index/indices parameter."
      }
    ],
    "type": "illegal_argument_exception",
    "reason": "Alias action can only work on its applied index so don't accept index/indices parameter."
  },
  "status": 400
}

Once this happens I have to check every index explicitly with the explain endpoint and look for this exception. Afterwards I need to remove the policy and re-apply it. This mostly happens when I reduce plugins.index_state_management.job_interval to less than 5.

How can one reproduce the bug?
Steps to reproduce the behaviour:
Currently, no known reproducer.

What is the expected behaviour?
If some sort of issues happens, the policy should fail and report it in its status and not prevent explaining the indices.

What is your host/environment?

  • OS: Opensearch Docker Image
  • Version 3.5.0
  • Plugins

Do you have any screenshots?
No

Do you have any additional context?
ISM Policy:

{
    "id": "archive-delete-365d-inst_kubernetes",
    "seqNo": 97068586,
    "primaryTerm": 42,
    "policy": {
        "policy_id": "archive-delete-365d-inst_kubernetes",
        "description": "Move index to warm node, delete after 365d",
        "last_updated_time": 1761836766371,
        "schema_version": 21,
        "error_notification": {
            "channel": {
                "id": "7JTtCIcBsTM5I_w0t4Yb"
            },
            "message_template": {
                "source": "The index {{ctx.index}} failed during policy execution.",
                "lang": "mustache"
            }
        },
        "default_state": "move",
        "states": [
            {
                "name": "move",
                "actions": [
                    {
                        "retry": {
                            "count": 3,
                            "backoff": "exponential",
                            "delay": "1m"
                        },
                        "replica_count": {
                            "number_of_replicas": 1
                        }
                    },
                    {
                        "retry": {
                            "count": 3,
                            "backoff": "exponential",
                            "delay": "1m"
                        },
                        "allocation": {
                            "require": {},
                            "include": {
                                "data": "warm"
                            },
                            "exclude": {},
                            "wait_for": false
                        }
                    },
                    {
                        "retry": {
                            "count": 3,
                            "backoff": "exponential",
                            "delay": "1m"
                        },
                        "read_only": {}
                    },
                    {
                        "retry": {
                            "count": 3,
                            "backoff": "exponential",
                            "delay": "1m"
                        },
                        "alias": {
                            "actions": [
                                {
                                    "add": {
                                        "aliases": [
                                            "inst_kubernetes_services-read"
                                        ]
                                    }
                                }
                            ]
                        }
                    }
                ],
                "transitions": [
                    {
                        "state_name": "delete",
                        "conditions": {
                            "min_index_age": "356d"
                        }
                    }
                ]
            },
            {
                "name": "delete",
                "actions": [
                    {
                        "retry": {
                            "count": 3,
                            "backoff": "exponential",
                            "delay": "1m"
                        },
                        "delete": {}
                    }
                ],
                "transitions": []
            }
        ],
        "ism_template": [
            {
                "index_patterns": [
                    "inst_kubernetes_services*-archive"
                ],
                "priority": 101,
                "last_updated_time": 1761836766371
            }
        ]
    }
}

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions