Add support for project_routing for _search and _async_search

[pawankartik-elastic]

project_routing can either be a query parameter or be included in the request's body:

{
  "project_routing": "_alias:_origin",
  "sort": [
    { "date": { "order": "asc" } }
  ],
  "aggs": {
    "sale_date": {
      "date_histogram": {
        "field": "date",
        "calendar_interval": "1d"
      }
    }
  }
}

At no time should it be set at both places. This PR adds support for it and the subsequent checks.

• Check if project_routing needs to be serialised/de-serialised.

[pawankartik-elastic]

Other endpoints use SearchRequest, too. Should we prohibit them from using project_routing? At the moment, we only look at whether CPS is enabled or not, not if an endpoint supports CPS, so I'm wondering about those endpoints, such as Fleet's.

The query parameter is consumed in the respective Rest*Action-s, so that's fine. It's the presence of the routing parameter in the request's body that I'm thinking about.

[piergm]

Also search with PIT id should not read/consume project_routing but that's with the PIT changes to make sure it doesn't happen, so we should be good here

[elasticsearchmachine]

Pinging @elastic/es-search-foundations (Team:Search Foundations)

[elasticsearchmachine]

Hi @pawankartik-elastic, I've created a changelog YAML for you.

[piergm]

Changes looks good, left some minor, non-blocking questions I'd like your opinion on.

[piergm]

Maybe add a comment here for future maintainers that we are going to validate the project routing and maybe throw in validateSearchRequest if it's not a CPS context, otherwise would be strange to see it parsed ok/without errors and an error thrown afterwards IMO.

[pawankartik-elastic]

Ack, making the change in the next push.

[quux00]

Good progress!

Some questions and suggestions left.

[quux00]

@piergm - Can you help me understand why we want projectRouting to be passed into CrossProjectIndexResolutionValidator.validate? This method appears to be called only from Transport Actions, not the SecurityActionFilter and its sole use in the validate method is to help determine whether the index expression is qualified. Why would projectRouting being set mean that we have a qualified expression?

That doesn't seem right to me, but maybe I'm missing a use case here.

[pawankartik-elastic]

Why would projectRouting being set mean that we have a qualified expression?

Consider these indices: foo,bar. Normally, they'd be considered unqualified, and in the flat world, we're fine with wherever they exist. However, if project_routing is specified, say, _alias:_origin, it means we're interested in the results from the origin. So you need to read both the indices and the routing info together.

Here's how it works, using Search as an example: the SAF uses project routing to resolve the projects that are in scope. Say, the routing is _alias:p1. All non-p1 projects are filtered out via CrossProjectRoutingResolver#resolve(), and index expressions are expanded/rewritten using these "resolved" projects. When the coordinator fans out the request, it only does it to those projects that are in scope wrt the routing info, i.e. p1 in this case. So the indices effectively belong to p1, and reading both the unqualified index expressions and the routing together, they can be treated as qualified.

[quux00]

Good progress. Two suggested changes.

[quux00]

Once the new TransportVersion and serialized projectRouting is done, LGTM, so approving now.