doc/cephadm: adding documentation for mgmt-gateway HA setup

Signed-off-by: Redouane Kachach <[email protected]>

Author: rkachach

doc/cephadm/services/mgmt-gateway.rst

@@ -49,6 +49,55 @@ monitoring `mgmt-gateway` takes care of handling HA when several instances of Pr
 available. The reverse proxy will automatically detect healthy instances and use them to process user requests.
 
 
+High Availability for mgmt-gateway service
+==========================================
+
+In addition to providing high availability for the underlying backend services, the mgmt-gateway
+service itself can be configured for high availability, ensuring that the system remains resilient
+even if certain core components for the service fail.
+
+Multiple mgmt-gateway instances can be deployed in an active/standby configuration using keepalived
+for seamless failover. The `oauth2-proxy` service can be deployed as multiple stateless instances,
+with nginx acting as a load balancer across them using round-robin strategy. This setup removes
+single points of failure and enhances the resilience of the entire system.
+
+In this setup, the underlying internal services follow the same high availability mechanism. Instead of
+directly accessing the `mgmt-gateway` internal endpoint, services use the virtual IP specified in the spec.
+This ensures that the high availability mechanism for `mgmt-gateway` is transparent to other services.
+
+Example Configuration for High Availability
+
+To deploy the mgmt-gateway in a high availability setup, here is an example of the specification files required:
+
+`mgmt-gateway` Configuration:
+
+.. code-block:: yaml
+
+    service_type: mgmt-gateway
+    placement:
+      label: mgmt
+    spec:
+      enable_auth: true
+      virtual_ip: 192.168.100.220
+
+`Ingress` Configuration for Keepalived:
+
+.. code-block:: yaml
+
+    service_type: ingress
+    service_id: ingress-mgmt-gw
+    placement:
+      label: mgmt
+    virtual_ip: 192.168.100.220
+    backend_service: mgmt-gateway
+    keepalive_only: true
+
+The number of deployed instances is determined by the number of hosts with the mgmt label.
+The ingress is configured in `keepalive_only` mode, with labels ensuring that any changes to
+the mgmt-gateway daemons are replicated to the corresponding keepalived instances. Additionally,
+the `virtual_ip` parameter must be identical in both specifications.
+
+
 Accessing services with mgmt-gateway
 ====================================
 
@@ -123,9 +172,6 @@ The specification can then be applied by running the following command:
 Limitations
 ===========
 
-A non-exhaustive list of important limitations for the mgmt-gateway service follows:
-
-* High-availability configurations and clustering for the mgmt-gateway service itself are currently not supported.
 * Services must bind to the appropriate ports based on the applications being proxied. Ensure that there
   are no port conflicts that might disrupt service availability.
 

doc/cephadm/services/oauth2-proxy.rst

@@ -42,8 +42,10 @@ a secure and flexible authentication mechanism.
 
 High availability
 ==============================
-`oauth2-proxy` is designed to integrate with an external IDP hence login high availability is not the responsibility of this
-service. In squid release high availability for the service itself is not supported yet.
+In general, `oauth2-proxy` is used in conjunction with the `mgmt-gateway`. The `oauth2-proxy` service can be deployed as multiple
+stateless instances, with the `mgmt-gateway` (nginx reverse-proxy) handling load balancing across these instances using a round-robin strategy.
+Since oauth2-proxy integrates with an external identity provider (IDP), ensuring high availability for login is managed externally
+and not the responsibility of this service.
 
 
 Accessing services with oauth2-proxy
@@ -70,8 +72,7 @@ An `oauth2-proxy` service can be applied using a specification. An example in YA
     service_type: oauth2-proxy
     service_id: auth-proxy
     placement:
-      hosts:
-        - ceph0
+      label: mgmt
     spec:
      https_address: "0.0.0.0:4180"
      provider_display_name: "My OIDC Provider"