Reject RECONNECT_UNLESS_SKIP_UNAVAILABLE in CPS

[JeremyDahlgren]

DisconnectedStrategy.RECONNECT_UNLESS_SKIP_UNAVAILABLE is unsupported in CPS, as skip_unavailable itself is unsupported in CPS. This change adds a check in RemoteClusterService.getRemoteClusterClient() rejecting RECONNECT_UNLESS_SKIP_UNAVAILABLE if stateless is enabled and adds a unit test case for it. This PR is a split from work initially done in #132478.

Resolves: ES-12610

[elasticsearchmachine]

Pinging @elastic/es-distributed-coordination (Team:Distributed Coordination)

[ywangd]

I wonder whether this change is absolutely necessary. Please correct me if I am wrong: technically it feels OK to use the RECONNECT_UNLESS_SKIP_UNAVAILABLE strategy in stateless? Since stateless has isSkipUnavailable fixed to true all the time, using this strategy means "do not attempt to reconnect". This seems semantically correct to me. Of course if the callers want to reconnect, a different strategy needs to be used and that is up to the caller. So the actual constraint here is the fixed isSkipUnavailable value. If this is well understood by both us and the search side, it seems OK that we can just leave the strategy as is?

[ywangd]

We talked about deferring this change until search side has made its changes. But I guess it's OK to proceed here because the condition is guarded with isStateless?

[JeremyDahlgren]

I wonder whether this change is absolutely necessary. Please correct me if I am wrong: technically it feels OK to use the RECONNECT_UNLESS_SKIP_UNAVAILABLE strategy in stateless? Since stateless has isSkipUnavailable fixed to true all the time, using this strategy means "do not attempt to reconnect". This seems semantically correct to me. Of course if the callers want to reconnect, a different strategy needs to be used and that is up to the caller. So the actual constraint here is the fixed isSkipUnavailable value. If this is well understood by both us and the search side, it seems OK that we can just leave the strategy as is?

Yes, we can close this out, Pawan is aligned here too, we talked about this when he was working on #132927.