Merge pull request ceph#57535 from rkachach/fix_issue_66095

Adding support for ceph mgmt-gateway

Reviewed-by: Adam King <[email protected]>
Reviewed-by: Anthony D'Atri <[email protected]>

Author: adk3798

doc/cephadm/services/index.rst

@@ -20,6 +20,7 @@ for details on individual services:
     snmp-gateway
     tracing
     smb
+    mgmt-gateway
 
 Service Status
 ==============

doc/cephadm/services/mgmt-gateway.rst

@@ -0,0 +1,150 @@
+.. _deploy-cephadm-mgmt-gateway:
+
+==================
+Management Gateway
+==================
+
+Deploying mgmt-gateway
+======================
+
+In Ceph releases beginning with Squid, the `mgmt-gateway` service introduces a new design for Ceph applications
+based on a modular, service-based architecture. This service, managed by cephadm and built on top of nginx
+(an open-source, high-performance web server), acts as the new front-end and single entry point to the
+Ceph cluster. The `mgmt-gateway` provides unified access to all Ceph applications, including the Ceph dashboard
+and monitoring stack. Employing nginx enhances security and simplifies access management due to its robust
+community support and high-security standards. The `mgmt-gateway` service acts as a reverse proxy that routes
+requests to the appropriate Ceph application instances.
+
+In order to deploy the mgmt-gateway service, use the following command:
+
+.. prompt:: bash #
+
+    ceph orch apply mgmt-gateway [--placement ...] ...
+
+Once applied cephadm will reconfigure specific running daemons (such as monitoring) to run behind the
+new created service. External access to those services will not be possible anymore. Access will be
+consolidated behind the new service endpoint: `https://<node-ip>:<port>`.
+
+
+Benefits of the mgmt-gateway service
+====================================
+* ``Unified Access``: Consolidated access through nginx improves security and provide a single entry point to services.
+* ``Improved user experience``: User no longer need to know where each application is running (ip/host).
+* ``High Availability for dashboard``: nginx HA mechanisms are used to provide high availability for the Ceph dashboard.
+* ``High Availability for monitoring``: nginx HA mechanisms are used to provide high availability for monitoring.
+
+Security enhancements
+=====================
+
+Once the `mgmt-gateway` service is deployed user cannot access monitoring services without authentication through the
+Ceph dashboard.
+
+
+High availability enhancements
+==============================
+nginx HA mechanisms are used to provide high availability for all the Ceph management applications including the Ceph dashboard
+and monitoring stack. In case of the Ceph dashboard user no longer need to know where the active manager is running.
+`mgmt-gateway` handles manager failover transparently and redirects the user to the active manager. In case of the
+monitoring `mgmt-gateway` takes care of handling HA when several instances of Prometheus, Alertmanager or Grafana are
+available. The reverse proxy will automatically detect healthy instances and use them to process user requests.
+
+
+Accessing services with mgmt-gateway
+====================================
+
+Once the `mgmt-gateway` service is deployed direct access to the monitoring services will not be allowed anymore.
+Applications including: Prometheus, Grafana and Alertmanager are now accessible through links
+from `Administration > Services`.
+
+
+Service Specification
+=====================
+
+A mgmt-gateway service can be applied using a specification. An example in YAML follows:
+
+.. code-block:: yaml
+
+    service_type: mgmt-gateway
+    service_id: gateway
+    placement:
+      hosts:
+        - ceph0
+    spec:
+     port: 5000
+     ssl_protocols:
+       - TLSv1.2
+       - TLSv1.3
+       - ...
+     ssl_ciphers:
+       - AES128-SHA
+       - AES256-SHA
+       - ...
+     ssl_certificate: |
+       -----BEGIN CERTIFICATE-----
+       MIIDtTCCAp2gAwIBAgIYMC4xNzc1NDQxNjEzMzc2MjMyXzxvQ7EcMA0GCSqGSIb3
+       DQEBCwUAMG0xCzAJBgNVBAYTAlVTMQ0wCwYDVQQIDARVdGFoMRcwFQYDVQQHDA5T
+       [...]
+       -----END CERTIFICATE-----
+    ssl_certificate_key: |
+       -----BEGIN PRIVATE KEY-----
+       MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQC5jdYbjtNTAKW4
+       /CwQr/7wOiLGzVxChn3mmCIF3DwbL/qvTFTX2d8bDf6LjGwLYloXHscRfxszX/4h
+       [...]
+       -----END PRIVATE KEY-----
+
+Fields specific to the ``spec`` section of the mgmt-gateway service are described below.
+
+.. py:currentmodule:: ceph.deployment.service_spec
+
+.. autoclass:: MgmtGatewaySpec
+   :members:
+
+.. warning::
+
+   TLSv1.3 is considered safe at this moment and includes a set of secure ciphers by default.
+   When configuring SSL/TLS ciphers for older versions, especially TLSv1.2, it is crucial to
+   use only a subset of secure ciphers. Using weak or outdated ciphers can significantly
+   compromise the security of your system.
+
+   Any alteration of the cipher list for SSL/TLS configurations is the responsibility of the
+   system administrator. Avoid modifying these lists without a thorough understanding of the
+   implications. Incorrect configurations can lead to vulnerabilities such as weak encryption,
+   lack of forward secrecy, and susceptibility to various attacks. Always refer to up-to-date
+   security guidelines and best practices when configuring SSL/TLS settings.
+
+
+The specification can then be applied by running the following command:
+
+.. prompt:: bash #
+
+   ceph orch apply -i mgmt-gateway.yaml
+
+
+Limitations
+===========
+
+A non-exhaustive list of important limitations for the mgmt-gateway service follows:
+
+* High-availability configurations and clustering for the mgmt-gateway service itself are currently not supported.
+* Services must bind to the appropriate ports based on the applications being proxied. Ensure that there
+  are no port conflicts that might disrupt service availability.
+
+
+Default images
+~~~~~~~~~~~~~~
+
+The `mgmt-gateway` service internally makes use of nginx reverse proxy. The following container image is used by default:
+
+::
+
+    DEFAULT_NGINX_IMAGE = 'quay.io/ceph/nginx:1.26.1'
+
+Admins can specify the image to be used by changing the `container_image_nginx` cephadm module option. If there were already
+running daemon(s) you must redeploy the daemon(s) in order to have them actually use the new image.
+
+For example:
+
+.. code-block:: bash
+
+     ceph config set mgr mgr/cephadm/container_image_nginx <new-nginx-image>
+     ceph orch redeploy mgmt-gateway

src/cephadm/cephadm.py

@@ -176,6 +176,7 @@
     NFSGanesha,
     SMB,
     SNMPGateway,
+    MgmtGateway,
     Tracing,
     NodeProxy,
 )
@@ -227,6 +228,7 @@ def get_supported_daemons():
     supported_daemons.append(Keepalived.daemon_type)
     supported_daemons.append(CephadmAgent.daemon_type)
     supported_daemons.append(SNMPGateway.daemon_type)
+    supported_daemons.append(MgmtGateway.daemon_type)
     supported_daemons.extend(Tracing.components)
     supported_daemons.append(NodeProxy.daemon_type)
     supported_daemons.append(SMB.daemon_type)
@@ -463,6 +465,8 @@ def update_default_image(ctx: CephadmContext) -> None:
             ctx.image = Keepalived.default_image
         if type_ == SNMPGateway.daemon_type:
             ctx.image = SNMPGateway.default_image
+        if type_ == MgmtGateway.daemon_type:
+            ctx.image = MgmtGateway.default_image
         if type_ == CephNvmeof.daemon_type:
             ctx.image = CephNvmeof.default_image
         if type_ in Tracing.components:
@@ -855,6 +859,10 @@ def create_daemon_dirs(
         sg = SNMPGateway.init(ctx, fsid, ident.daemon_id)
         sg.create_daemon_conf()
 
+    elif daemon_type == MgmtGateway.daemon_type:
+        cg = MgmtGateway.init(ctx, fsid, ident.daemon_id)
+        cg.create_daemon_dirs(data_dir, uid, gid)
+
     elif daemon_type == NodeProxy.daemon_type:
         node_proxy = NodeProxy.init(ctx, fsid, ident.daemon_id)
         node_proxy.create_daemon_dirs(data_dir, uid, gid)
@@ -3571,6 +3579,9 @@ def list_daemons(
                                 elif daemon_type == SNMPGateway.daemon_type:
                                     version = SNMPGateway.get_version(ctx, fsid, daemon_id)
                                     seen_versions[image_id] = version
+                                elif daemon_type == MgmtGateway.daemon_type:
+                                    version = MgmtGateway.get_version(ctx, container_id)
+                                    seen_versions[image_id] = version
                                 else:
                                     logger.warning('version for unknown daemon type %s' % daemon_type)
                         else:

src/cephadm/cephadmlib/constants.py

@@ -19,6 +19,7 @@
 DEFAULT_JAEGER_AGENT_IMAGE = 'quay.io/jaegertracing/jaeger-agent:1.29'
 DEFAULT_JAEGER_QUERY_IMAGE = 'quay.io/jaegertracing/jaeger-query:1.29'
 DEFAULT_SMB_IMAGE = 'quay.io/samba.org/samba-server:devbuilds-centos-amd64'
+DEFAULT_NGINX_IMAGE = 'quay.io/ceph/nginx:1.26.1'
 DEFAULT_REGISTRY = 'docker.io'  # normalize unqualified digests to this
 # ------------------------------------------------------------------------------
 
@@ -49,3 +50,5 @@
 DATEFMT = '%Y-%m-%dT%H:%M:%S.%fZ'
 QUIET_LOG_LEVEL = 9  # DEBUG is 10, so using 9 to be lower level than DEBUG
 NO_DEPRECATED = False
+UID_NOBODY = 65534
+GID_NOGROUP = 65534

src/cephadm/cephadmlib/daemons/__init__.py

@@ -9,6 +9,7 @@
 from .snmp import SNMPGateway
 from .tracing import Tracing
 from .node_proxy import NodeProxy
+from .mgmt_gateway import MgmtGateway
 
 __all__ = [
     'Ceph',
@@ -25,4 +26,5 @@
     'SNMPGateway',
     'Tracing',
     'NodeProxy',
+    'MgmtGateway',
 ]

src/cephadm/cephadmlib/daemons/custom.py

@@ -15,6 +15,7 @@
 from ..deployment_utils import to_deployment_container
 from ..file_utils import write_new, makedirs
 from ..net_utils import EndPoint
+from ..constants import UID_NOBODY, GID_NOGROUP
 
 
 logger = logging.getLogger()
@@ -43,8 +44,8 @@ def __init__(
 
         # config-json options
         self.entrypoint = dict_get(config_json, 'entrypoint')
-        self.uid = dict_get(config_json, 'uid', 65534)  # nobody
-        self.gid = dict_get(config_json, 'gid', 65534)  # nobody
+        self.uid = dict_get(config_json, 'uid', UID_NOBODY)
+        self.gid = dict_get(config_json, 'gid', GID_NOGROUP)
         self.volume_mounts = dict_get(config_json, 'volume_mounts', {})
         self.args = dict_get(config_json, 'args', [])
         self.envs = dict_get(config_json, 'envs', [])