TransportClusterSearchShardsAction runs locally

[burqen]

This action only need cluster state and can be run any node. Make sure to keep necessary serialization and deserialization for BwC.

Relates #101805

[burqen]

I'm not sure what :team and area this PR belongs to. Any ideas?

[elasticsearchmachine]

Pinging @elastic/es-distributed-coordination (Team:Distributed Coordination)

[DaveCTurner]

Ok the tests seem very unhappy because we use this request on remote clusters. The change to a LocalClusterStateRequest is still valid because we want the handling node in the remote cluster to respond directly rather than forwarding the response on to its own master node, but we will have to continue supporting sending this action over the wire.

We'll need to make a separate org.elasticsearch.action.RemoteClusterActionType to distinguish the cluster-local action from the one that works on remote clusters, and also find a way to distinguish the local request (which rejects writeTo() calls) from the request that gets sent to the remote cluster.

[DaveCTurner]

No need for the AP - we can use git annotate to find you if we need to :)

[burqen]

After offline discussion with @DaveCTurner I've concluded that this task is much too involved for a first PR and I will close it now.
One way that we could have gone about it would be to have two separate TransportClusterSearchShardActions, one for cross cluster and one for local execution. But that's left for another day.