[DOCS] Remote cluster troubleshooting guide (#99128) (#99199)

* [DOCS] Remote cluster troubleshooting guide

* Fix test failures

* Apply suggestions from code review



* Review feedback

* Group issues under 'common' and 'API key'

* Apply suggestions from code review



---------

Co-authored-by: Yang Wang <[email protected]>

Author: abdonpijpelink

docs/reference/modules/cluster/remote-clusters-api-key.asciidoc

@@ -48,6 +48,8 @@ To add a remote cluster using API key authentication:
 . <<remote-clusters-connect-api-key>>
 . <<remote-clusters-privileges-api-key>>
 
+If you run into any issues, refer to <<remote-clusters-troubleshooting>>.
+
 [[remote-clusters-prerequisites-api-key]]
 ==== Prerequisites
 

docs/reference/modules/cluster/remote-clusters-cert.asciidoc

@@ -8,6 +8,8 @@ To add a remote cluster using TLS certificate authentication:
 . <<remote-clusters-connect-cert>>
 . <<remote-clusters-privileges-cert>>
 
+If you run into any issues, refer to <<remote-clusters-troubleshooting>>.
+
 [[remote-clusters-prerequisites-cert]]
 ==== Prerequisites
 

docs/reference/modules/cluster/remote-clusters-migration.asciidoc

@@ -32,6 +32,8 @@ following these steps:
 . <<remote-clusters-migration-resume>>
 . <<remote-clusters-migration-disable-cert>>
 
+If you run into any issues, refer to <<remote-clusters-troubleshooting>>.
+
 [[remote-clusters-migration-prerequisites]]
 ==== Prerequisites
 
@@ -208,11 +210,10 @@ existing indices that were created from the auto-follow pattern.
 [[remote-clusters-migration-disable-cert]]
 ==== Disable certificate based authentication and authorization
 
-//TODO: Add link to troubleshooting docs when they're published
 NOTE: Only proceed with this step if the migration has been proved successful on
-the local cluster. If the migration is unsuccessful, either find out what the
-problem is and attempt to fix it or <<remote-clusters-migration-rollback,roll
-back>>.
+the local cluster. If the migration is unsuccessful, either
+<<remote-clusters-troubleshooting,find out what the problem is and attempt to
+fix it>> or <<remote-clusters-migration-rollback,roll back>>.
 
 Next, disable the certification based connection. Optionally, you can also
 revoke the authorization.

docs/reference/modules/cluster/remote-clusters-remote-info.asciidoc

@@ -0,0 +1,32 @@
+[source,console]
+----
+GET /_remote/info
+----
+// TEST[skip:TODO]
+
+The API should return `"connected" : true`. When using
+<<remote-clusters-api-key,API key authentication>>, it should also return
+`"cluster_credentials": "::es_redacted::"`.
+
+[source,console-result,subs=attributes+]
+----
+{
+  "cluster_one" : {
+    "seeds" : [
+      "127.0.0.1:9443"
+    ],
+    "connected" : true, <1>
+    "num_nodes_connected" : 1, 
+    "max_connections_per_cluster" : 3,
+    "initial_connect_timeout" : "30s",
+    "skip_unavailable" : false,
+    "cluster_credentials": "::es_redacted::", <2>
+    "mode" : "sniff"
+  }
+}
+----
+// TEST[skip:TODO]
+<1> The remote cluster has connected successfully.
+<2> If present, indicates the remote cluster has connected using
+<<remote-clusters-api-key,API key authentication>> instead of
+<<remote-clusters-cert,certificate based authentication>>.

docs/reference/modules/cluster/remote-clusters-troubleshooting.asciidoc

@@ -97,3 +97,5 @@ include::cluster/remote-clusters-cert.asciidoc[]
 include::cluster/remote-clusters-migration.asciidoc[]
 
 include::cluster/remote-clusters-settings.asciidoc[]
+
+include::cluster/remote-clusters-troubleshooting.asciidoc[]

docs/reference/modules/remote-clusters.asciidoc

@@ -55,7 +55,8 @@ fix problems that an {es} deployment might encounter.
 * <<transform-troubleshooting,Troubleshooting transforms>>
 * <<watcher-troubleshooting,Troubleshooting Watcher>>
 * <<troubleshooting-searches,Troubleshooting searches>>
-* <<troubleshooting-shards-capacity-issues, Troubleshoot shards capacity>>
+* <<troubleshooting-shards-capacity-issues,Troubleshooting shards capacity>>
+* <<remote-clusters-troubleshooting,Troubleshooting remote clusters>>
 
 If none of these solutions relate to your issue, you can still get help: