[doc] Update how to handle trusted certificates when pool.join

minglumlu · minglumlu · commit 36890b698383 · 2026-02-04T15:30:19.000+08:00
Signed-off-by: Ming Lu &lt;ming.lu@cloud.com&gt;
diff --git a/doc/content/design/pool-certificates.md b/doc/content/design/pool-certificates.md
@@ -6,6 +6,8 @@ revision: 2
 status: released (22.6.0)
 ---
 
+This design is modified by [trusted-certificates.md](trusted-certificates.md).
+
 ## Overview
 
 Xenserver has used TLS-encrypted communications between xapi daemons in a pool since its first release.
@@ -353,22 +355,30 @@ This feature needs clients to behave differently when initiating pool joins, to
 Several alerts are introduced:
 * POOL_CA_CERTIFICATE_EXPIRING_30, POOL_CA_CERTIFICATE_EXPIRING_14, POOL_CA_CERTIFICATE_EXPIRING_07, POOL_CA_CERTIFICATE_EXPIRED: Similar to host certificates, now the user-installable pool's CA certificates are monitored for expiry dates and alerts are generated about them. The body for this type of message is:
 
+```
     <body><message>The trusted TLS server certificate {is expiring soon|has expired}.</message><date>20210302T02:00:01Z</date></body>
+```
 
 * HOST_INTERNAL_CERTIFICATE_EXPIRING_30, HOST_INTERNAL_CERTIFICATE_EXPIRING_14, HOST_INTERNAL_CERTIFICATE_EXPIRING_07, HOST_INTERNAL_CERTIFICATE_EXPIRED: Similar to host certificates, the newly-introduced hosts' internal server certificates are monitored for expiry dates and alerts are generated about them. The body for this type of message is:
 
+```
     <body><message>The TLS server certificate for internal communications {is expiring soon|has expired}.</message><date>20210302T02:00:01Z</date></body>
+```
 
 * TLS_VERIFICATION_EMERGENCY_DISABLED: The host is in emergency mode and is not enforcing tls verification anymore, the situation that forced the disabling must be fixed and the verification enabled ASAP.
 
+```
     <body><host>HOST-UUID</host></body>
+```
 
 * FAILED_LOGIN_ATTEMPTS: An hourly alert that contains the number of failed attempts and the 3 most common origins for these failed alerts. The body for this type of message is:
 
+```
     <body>
     <total>35</total>
     <known><username>usr5</username><originator>origin5</originator><ip>5.4.3.2</ip><number>10</number><date>20200922T15:03:13Z</date></known>
     <known><username>usr4</username><useragent>UA</useragent><number>6</number><date>20200922T15:03:13Z</date></known>
     <known><useragent>UA</useragent><ip>4.3.2.1</ip><number>4</number><date>20200922T14:57:11Z</date></known>
     <unknown>10</unknown>
     </body>
+```
diff --git a/doc/content/design/trusted-certificates.md b/doc/content/design/trusted-certificates.md
@@ -2,16 +2,16 @@
 title: Trusted certificates for identity validation in TLS connections
 layout: default
 design_doc: true
-revision: 1
+revision: 2
 status: draft
 ---
 
 # Overview
 
 In various use cases, TLS connections are established on the host on which XAPI runs.
 When establishing a TLS connection, the peer identity needs to be validated.
-This is done using either a root CA certificate to perform certificate chain validation, or a known peer certificate for validation with certificate pinning.
-The root CA certificates and peer certificates involved in this process are referred to as trusted certificates.
+This is done using either a root CA certificate to perform certificate chain validation, or a pinned certificate for validation with certificate pinning.
+The root CA certificates and pinned certificates involved in this process are referred to as trusted certificates.
 When a trusted certificate is installed, the local endpoint can validate the peer identity during TLS connection establishment.
 Certificate chain validation is a general-purpose, standards-based approach but requires additional steps, such as getting the peer's certificate signed by a CA.
 In contrast, certificate pinning offers a quicker way to set up trust in some cases without the overhead of CA signing.
@@ -21,13 +21,13 @@ This allows the use case to start in quicker and easier way without prior CA sig
 
 As the unified API for the whole system, XAPI also exposes interfaces for users to install and manage trusted certificates that are used by system components for different purposes.
 
-The base design described in [pool-certificates.md](https://github.com/minglumlu/xen-api/blob/5d1ea1520825d502c57a90a02db476cd7d6a9132/doc/content/design/pool-certificates.md) defines the database, API, and trust store in the filesystem for managing trusted certificates.
+The base design described in [pool-certificates.md](pool-certificates.md) defines the database, API, and trust store in the filesystem for managing trusted certificates.
 This document introduces the following enhancements to that design:
 
-* Explicit separation of root CA certificates and peer certificates:
+* Explicit separation of root CA certificates and pinned certificates:
 In the base design, both certificate types share the same database schema, APIs, and are stored together in a single bundle file.
 This makes it difficult to determine the appropriate validation approach based on the certificate type.
-The improvement introduces a type value to separate root CA certificates and peer certificates explicitly.
+The improvement introduces a type value to separate root CA certificates and pinned certificates explicitly.
 
 * Add a "purpose" attribute for trusted certificates:
 According to the base design, only certificates used for internal TLS connections among XAPI processes within a pool are stored separately.
@@ -49,13 +49,13 @@ This case benefits from the improvements introduced in this design as well.
 ## Database schema
 The *Certificate* class in database is defined to represent general certificates, including trusted certificates.
 One existing class field "type" supports the following enumeration values:
-* "ca": trusted certificates including both root CA and peer.
+* "ca": trusted certificates including both root CA and pinned.
 * "host": identity certificate of a host for communication with entities outside the pool.
 * "host_internal": identity certificate of a host for communication with other pool members.
 
 Two improvements in this design:
-* A new value "peer" is introduced in this design so that the existing "ca" now represents trusted root CA only.
-The new "peer" will represent trusted peer certificates.
+* A new value "pinned" is introduced in this design so that the existing "ca" now represents trusted root CA only.
+The new "pinned" will represent trusted pinned certificates.
 
 * A new enumeration type "purpose" is introduced to indicate the intended usage of a trusted certificate.
 A new *Certificate* class field "purpose" (a set of values of enumeration type "purpose") will be added to represent all applicable purposes of a trusted certificate.
@@ -79,27 +79,122 @@ For the same reason, "pool.uninstall_ca_certificate" will also be deprecated.
 This is a new API introduced in this design with its arguments being defined as:
 * session (ref session_id): reference to a valid session;
 * self (ref Pool): reference to the pool;
-* ca (boolean): the trusted certificate is a root CA certificate used to verify a chain (true), or a peer certificate used for certificate pinning (false);
+* ca (boolean): the trusted certificate is a root CA certificate used to verify a chain (true), or a pinned certificate used for certificate pinning (false);
 * cert (string): the trusted certificate in PEM format;
 * purpose (string list): the purposes of the trusted certificate.
 
 This new API is used to install trusted certificate.
 When *purpose* is an empty set, it stands for a root CA certificate for general purpose.
-The *purpose* can not be an empty set when the *ca* is false, because each peer certificate is specific to a single server and therefore unsuitable for a shared trusted certificate for general purpose.
+The *purpose* can not be an empty set when the *ca* is false, because each pinned certificate is specific to a single server and therefore unsuitable for a shared trusted certificate for general purpose.
 
 It returns *void* when succeed. Otherwise, return corresponding API error.
 
 ### pool.uninstall_trusted_certificate
 This is a new API introduced in this design to uninstall a trusted certificate with its arguments being defined as:
 * session (ref session_id): reference to a valid session;
 * certificate (ref Certificate): reference to the trusted certificate;
-* force (bool): remove the database entry even if the file doesn't exist.
 
 It returns *void* when succeed. Otherwise, return corresponding API error.
 
 ### pool.join
-Prior to this design, trusted certificates are exchanged between the pool and the joining host during the pre‑join phase.
-This design preserves that behavior to ensure the joiner works correctly both before and after joining the pool.
+According to the base design, trusted certificates are exchanged between the pool and the joining host during the pre‑join phase.
+This design basically preserves that behavior to ensure the joiner works correctly both before and after joining the pool.
+However, a number of modifications have been introduced compared with the base design.
+
+~~~mermaid
+
+sequenceDiagram
+participant clnt as Client
+participant join as Joiner
+participant coor as Coordinator
+participant memb as Member
+clnt->>join: Pool.join coordinator_ip coordinator_username coordinator_password
+join->>coor:login_with_password rpc_no_verify coordinator_ip coordinator_username coordinator_password
+coor-->>join:
+
+Note over join: pre_join_checks
+rect rgba(0,0,0,0.05)
+join->>join: assert_tls_verification_matches
+alt fails
+Note over join: interrupt join, raise error
+end
+end
+
+Note over join: exchnage trusted intra-pool host identity certificates
+rect rgba(0,0,0,0.05)
+join->>coor: Pool.exchange_certificates_on_join <Joiner's trusted host identity cert>
+coor->>coor: Cert_distrib.exchange_certificates_with_joiner start
+coor->>memb: Host.cert_distrib_atom Write
+memb-->>coor:
+coor->>coor: Cert_distrib.get_local_pool_certs
+coor-->>coor: Cert_distrib.exchange_certificates_with_joiner done
+coor-->>join: <trusted host identity certs in pool>
+join->>join: Cert_distrib.import_joining_pool_certs <trusted host identity certs in pool>
+end
+
+join->>coor:login_with_password rpc_verify coordinator_ip coordinator_username coordinator_password
+coor-->>join:
+
+Note over join: exchange legacy ca certificates
+rect rgba(0,0,0,0.05)
+join->>coor: Pool.exchange_ca_certificates_on_join <Joiner's legacy ca certs>
+coor->>coor: Cert_distrib.exchange_ca_certificates_with_joiner
+coor-->>join: <legacy ca certs in pool>
+join->>join: Cert_distrib.import_joining_pool_ca_certificates <legacy ca certs in pool>
+end
+
+Note over join: exchange trusted certificates
+rect rgba(0,0,0,0.05)
+join->>coor: Pool.exchange_trusted_certificates_on_join <Joiner's trusted certs>
+loop for every <trusted> in Joiner
+coor->>coor: Pool.install_trusted_certificate start
+coor->>memb: Host.cert_distrib_atom Write
+memb-->>coor:
+coor->>memb: Host.certificate_sync
+memb-->>coor:
+coor-->>coor: Pool.install_trusted_certificate done
+end
+coor->>coor: Cert_distrib.collect_trusted_certs
+coor-->>join: <trusted certs in pool>
+loop for every <trusted> in pool
+join->>join: Pool.install_trusted_certificate
+end
+end
+
+Note over join: exchange CRLs
+rect rgba(0,0,0,0.05)
+join->>coor: Pool.exchange_crls_on_join <Joiner's CRLs>
+loop for every <CRL> in Joiner
+coor->>coor: Pool.crl_install start
+coor->>memb: Host.cert_distrib_atom Write
+memb-->>coor:
+coor->>memb: Host.certificate_sync
+memb-->>coor:
+coor-->>coor: Pool.crl_install done
+end
+coor->>coor: Cert_distrib.collect_crls
+coor-->>join: <CLRs in pool>
+loop for every <CRL> in pool
+join->>join: Pool.crl_install
+end
+end
+
+join->>coor: Host.add joiner
+coor-->>join:
+
+join->>join: restart_as_slave
+
+Note over join: Copy all certificates from coordinator
+rect rgba(0,0,0,0.05)
+join->>coor: Host.copy_primary_host_certs
+coor->>join: Host.cert_distrib_atom Write
+join-->>coor:
+coor->>join: Host.cert_distrib_atom GenBundle
+join-->>coor:
+coor-->>join: return from Host.copy_primary_host_certs
+end
+
+~~~
 
 ### pool.eject
 The trusted certificates will be removed from any host which is being eject from the pool.
@@ -130,10 +225,10 @@ The stores for the certificates installed via "pool.install_trusted_certificate"
 | Name | Filesystem location | Used for |
 | ---- | ------------------- | -------- |
 | Trusted General CA | /etc/trusted-certs/ca-general/     | Trusted root CA certificates that users can install to validate a peer’s identity when establishing a TLS connection for general purpose.
-| Trusted Peer | /etc/trusted-certs/peer-\<PURPOSE\>/     | Trusted peer certificates that users can install to validate a peer’s identity when establishing a TLS connection for \<PURPOSE\>.
+| Trusted Peer | /etc/trusted-certs/pinned-\<PURPOSE\>/     | Trusted pinned certificates that users can install to validate a peer’s identity when establishing a TLS connection for \<PURPOSE\>.
 | Trusted CA | /etc/trusted-certs/ca-\<PURPOSE\>/         | Trusted root CA certificates that users can install to validate a peer’s identity when establishing a TLS connection for \<PURPOSE\>.
 | General Bundle  | /etc/trusted-certs/ca-bundle-general.pem    | Bundle of trusted root CA certificates under /etc/trusted-certs/ca-general/ to verify a peer's identity when establishing a TLS connection for general purpose.
-| Peer Bundle  | /etc/trusted-certs/peer-bundle-\<PURPOSE\>.pem | Bundle of trusted peer certificates under /etc/trusted-certs/peer-\<PURPOSE\>/ to verify a peer's identity when establishing a TLS connection for \<PURPOSE\>.
+| Peer Bundle  | /etc/trusted-certs/pinned-bundle-\<PURPOSE\>.pem | Bundle of trusted pinned certificates under /etc/trusted-certs/pinned-\<PURPOSE\>/ to verify a peer's identity when establishing a TLS connection for \<PURPOSE\>.
 | CA Bundle     | /etc/trusted-certs/ca-bundle-\<PURPOSE\>.pem  | Bundle of trusted root CA certificates under /etc/trusted-certs/ca-\<PURPOSE\>/ to verify a peer's identity when establishing a TLS connection for \<PURPOSE\>.
 
 The filesystem location is derived from the \<PURPOSE\>. Each \<PURPOSE\> string corresponds to a predefined value of the "purpose" type in the database, implemented as predefined constants.
