Synonyms API: Index gets red status when synonyms set does not exist

[carlosdelest]

Elasticsearch Version

8.10+

Installed Plugins

No response

Java Version

bundled

OS Version

any

Problem Description

It is possible to create an index with an analyzer that uses a synonym or synonym_graph filter, that point to a synonym set that does not exist.

When that happens, the index is created but the shards cannot be recovered (as synonyms are loaded during shard recovery). This makes both the index and the cluster health to be red, as no shard can be recovered for it.

Users can solve the problem by creating the synonym set via the Synonyms API.

Steps to Reproduce

Create an index that uses a synonyms set that has not been created yet:

PUT test-synonyms
{
  "settings": {
    "analysis": {
      "filter": {
        "synonyms_filter": {
          "type": "synonym_graph",
          "synonyms_set": "my-synonyms-set",
          "updateable": true
        }
      },
      "analyzer": {
        "my_index_analyzer": {
          "type": "custom",
          "tokenizer": "standard",
          "filter": ["lowercase"]
        },
        "my_search_analyzer": {
          "type": "custom",
          "tokenizer": "standard",
          "filter": ["lowercase", "synonyms_filter"]
        }
      }
    }
  },
  "mappings": {
    "properties": {
      "title": {
        "type": "text",
        "analyzer": "my_index_analyzer",
        "search_analyzer": "my_search_analyzer"
      }
    }
  }
}

The index is created, but shards do not acknowledge:

{
    "acknowledged": true,
    "shards_acknowledged": false,
    "index": "test-synonyms"
}

The index and the cluster are in red status:

GET _cat/indices
    {
        "health": "red",
        "status": "open",
        "index": "test-synonyms",
        "uuid": "CFdqVx5XRsCvIpNGLEQHbw",
        "pri": "1",
        "rep": "1",
        "docs.count": null,
        "docs.deleted": null,
        "store.size": null,
        "pri.store.size": null,
        "dataset.size": null
    }

Logs (if relevant)

No response

[elasticsearchmachine]

Pinging @elastic/es-search-relevance (Team:Search Relevance)

[elasticsearchmachine]

Pinging @elastic/search-eng (Team:SearchOrg)

[elasticsearchmachine]

Pinging @elastic/search-relevance (Team:Search - Relevance)

[kderusso]

Hey @carlosdelest, this is related to #101693 - If I recall correctly we decided that documenting the workaround was the best solution at the time.

The workaround is documented here:

Synonyms sets must exist before they can be added to indices. If an index is created referencing a nonexistent synonyms set, the index will remain in a partially created and inoperable state. The only way to recover from this scenario is to ensure the synonyms set exists then either delete and re-create the index, or close and re-open the index.

The two options we'd thought about at the time were:

1. Completely fail to create the index
2. Be lenient and allow the offending synonym filter to act as a passthrough until the synonyms set is created. (I think I slightly prefer that solution)

Neither are non-trivial fixes.

[carlosdelest]

Hey @kderusso - sorry for the late reply, this slipped through the cracks.

This came back to attention because of serverless - users can have a red index doing regular actions.

This has been fixed via #125659, that uses a similar strategy than #110901 . It uses the lenient setting to ignore a non-existing synonyms set, and issue a log warning accordingly.