Skip to content

Creating index-patterns causes search threadPool to be exhausted #8596

@kkewwei

Description

@kkewwei

Is your feature request related to a problem? Please describe.

  1. when I set the ccs in the cluster settings(the address is pointed to itself to reproduce)
PUT _cluster/settings
{
  "persistent": {
      "cluster": {
         "remote": {
            "ccs1": {
               "mode": "proxy",
               "proxy_address": "127.0.0.1:9300"
            }
         }
      }
  }
}

and create a new index-patterns in the dashboard, I find that search thread pool is immediately exhausted in the target cluster:
image
And search slow log is as follows:
image

POST *:*/_search
{
  "size": 0,
  "timeout": "30000ms",
  "query": {
    "match_all": {
      "boost": 1
    }
  },
  "track_total_hits": 2147483647,
  "aggregations": {
    "indices": {
      "terms": {
        "field": "_index",
        "size": 200,
        "min_doc_count": 1,
        "shard_min_doc_count": 0,
        "show_term_doc_count_error": false,
        "order": [
          {
            "_count": "desc"
          },
          {
            "_key": "asc"
          }
        ]
      }
    }
  }
}
  1. As the total_shards=35720, which equals the total number shards of the cluster, we know that all the indices are aggregated by the query.

  2. We know that the query is from the "Index_Patterns".
    image

It seems that we send the query to get the indices list of target cluster, when the target cluster is very large, this query will become very dangerous.

Describe the solution you'd like
If we should add the api in the opensearch code core to get indices list of the ccs target cluster?

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions