Change master->primary, take two (patroni#3127)

This commit is a breaking change:
1. `role` in DCS is written as "primary" instead of "master".
2. `role` in REST API responses is also written as "primary".
3. REST API no longer accepts role=master in requests (for example switchover/failover/restart endpoints).
4. `/metrics` REST API endpoint will no longer report `patroni_master`.
5. `patronictl` no longer accepts `--master` argument.
6. `no_master` option in declarative configuration of custom replica creation methods is no longer treated as a special option, please use `no_leader` instead.
7. `patroni_wale_restore` doesn't accept `--no_master` anymore.
8. `patroni_barman` doesn't accept `--role=master` anymore.
9. callback scripts will be executed with role=primary instead of role=master
10. On Kubernetes Patroni by default will set role label to primary. In case if you want to keep old behavior and avoid downtime or lengthy complex migrations you can configure `kubernetes.leader_label_value` and `kubernetes.standby_leader_label_value` to `master`.

However, a few exceptions regarding master are still in place:
1. `GET /master` REST API endpoint will continue to work.
2. `master_start_timeout` and `master_stop_timeout` in global configuration are still accepted.
3. `master` tag is still preserved in Consul services in addition to `primary`.

Rationale for these exceptions: DBA doesn't always 100% control the infrastructure and can't adjust the configuration.

Author: CyberDem0n

docs/ENVIRONMENT.rst

@@ -55,7 +55,7 @@ Consul
 -  **PATRONI\_CONSUL\_CONSISTENCY**: (optional) Select consul consistency mode. Possible values are ``default``, ``consistent``, or ``stale`` (more details in `consul API reference <https://www.consul.io/api/features/consistency.html/>`__)
 -  **PATRONI\_CONSUL\_CHECKS**: (optional) list of Consul health checks used for the session. By default an empty list is used.
 -  **PATRONI\_CONSUL\_REGISTER\_SERVICE**: (optional) whether or not to register a service with the name defined by the scope parameter and the tag master, primary, replica, or standby-leader depending on the node's role. Defaults to **false**
--  **PATRONI\_CONSUL\_SERVICE\_TAGS**: (optional) additional static tags to add to the Consul service apart from the role (``master``/``primary``/``replica``/``standby-leader``). By default an empty list is used.
+-  **PATRONI\_CONSUL\_SERVICE\_TAGS**: (optional) additional static tags to add to the Consul service apart from the role (``primary``/``replica``/``standby-leader``). By default an empty list is used.
 -  **PATRONI\_CONSUL\_SERVICE\_CHECK\_INTERVAL**: (optional) how often to perform health check against registered url
 -  **PATRONI\_CONSUL\_SERVICE\_CHECK\_TLS\_SERVER\_NAME**: (optional) overide SNI host when connecting via TLS, see also `consul agent check API reference <https://www.consul.io/api-docs/agent/check#tlsservername>`__.
 
@@ -113,11 +113,11 @@ Kubernetes
 -  **PATRONI\_KUBERNETES\_NAMESPACE**: (optional) Kubernetes namespace where the Patroni pod is running. Default value is `default`.
 -  **PATRONI\_KUBERNETES\_LABELS**: Labels in format ``{label1: value1, label2: value2}``. These labels will be used to find existing objects (Pods and either Endpoints or ConfigMaps) associated with the current cluster. Also Patroni will set them on every object (Endpoint or ConfigMap) it creates.
 -  **PATRONI\_KUBERNETES\_SCOPE\_LABEL**: (optional) name of the label containing cluster name. Default value is `cluster-name`.
--  **PATRONI\_KUBERNETES\_ROLE\_LABEL**: (optional) name of the label containing role (master or replica or other custom value). Patroni will set this label on the pod it runs in. Default value is ``role``.
--  **PATRONI\_KUBERNETES\_LEADER\_LABEL\_VALUE**: (optional) value of the pod label when Postgres role is `master`. Default value is `master`.
+-  **PATRONI\_KUBERNETES\_ROLE\_LABEL**: (optional) name of the label containing role (`primary`, `replica` or other custom value). Patroni will set this label on the pod it runs in. Default value is ``role``.
+-  **PATRONI\_KUBERNETES\_LEADER\_LABEL\_VALUE**: (optional) value of the pod label when Postgres role is `primary`. Default value is `primary`.
 -  **PATRONI\_KUBERNETES\_FOLLOWER\_LABEL\_VALUE**: (optional) value of the pod label when Postgres role is `replica`. Default value is `replica`.
--  **PATRONI\_KUBERNETES\_STANDBY\_LEADER\_LABEL\_VALUE**: (optional) value of the pod label when Postgres role is ``standby_leader``. Default value is ``master``.
--  **PATRONI\_KUBERNETES\_TMP\_ROLE\_LABEL**: (optional) name of the temporary label containing role (master or replica). Value of this label will always use the default of corresponding role. Set only when necessary.
+-  **PATRONI\_KUBERNETES\_STANDBY\_LEADER\_LABEL\_VALUE**: (optional) value of the pod label when Postgres role is ``standby_leader``. Default value is ``primary``.
+-  **PATRONI\_KUBERNETES\_TMP\_ROLE\_LABEL**: (optional) name of the temporary label containing role (`primary` or `replica`). Value of this label will always use the default of corresponding role. Set only when necessary.
 -  **PATRONI\_KUBERNETES\_USE\_ENDPOINTS**: (optional) if set to true, Patroni will use Endpoints instead of ConfigMaps to run leader elections and keep cluster state.
 -  **PATRONI\_KUBERNETES\_POD\_IP**: (optional) IP address of the pod Patroni is running in. This value is required when `PATRONI_KUBERNETES_USE_ENDPOINTS` is enabled and is used to populate the leader endpoint subsets when the pod's PostgreSQL is promoted.
 -  **PATRONI\_KUBERNETES\_PORTS**: (optional) if the Service object has the name for the port, the same name must appear in the Endpoint object, otherwise service won't work. For example, if your service is defined as ``{Kind: Service, spec: {ports: [{name: postgresql, port: 5432, targetPort: 5432}]}}``, then you have to set ``PATRONI_KUBERNETES_PORTS='[{"name": "postgresql", "port": 5432}]'`` and Patroni will use it for updating subsets of the leader Endpoint. This parameter is used only if `PATRONI_KUBERNETES_USE_ENDPOINTS` is set.

docs/ha_loop_diagram.dot

@@ -109,7 +109,7 @@ digraph G {
   subgraph cluster_process_healthy_cluster {
     label = "process_healthy_cluster"
     "healthy_has_lock" [label="Am I the owner of the leader lock?", shape=diamond]
-    "healthy_is_leader" [label="Is Postgres running as master?", shape=diamond]
+    "healthy_is_leader" [label="Is Postgres running as primary?", shape=diamond]
     "healthy_no_lock" [label="Follow the leader (async,\ncreate/update recovery.conf and restart if necessary)"]
     "healthy_has_lock" -> "healthy_no_lock" [label="no" color="red"]
     "healthy_has_lock" -> "healthy_update_leader_lock" [label="yes" color="green"]
@@ -119,7 +119,7 @@ digraph G {
     "healthy_update_success" -> "healthy_is_leader" [label="yes" color="green"]
     "healthy_update_success" -> "healthy_demote" [label="no" color="red"]
     "healthy_demote" [label="Demote (async,\nrestart in read-only)"]
-    "healthy_failover" [label="Promote Postgres to master"]
+    "healthy_failover" [label="Promote Postgres to primary"]
     "healthy_is_leader" -> "healthy_failover" [label="no" color="red"]
   }
   "healthy_demote" -> "update_member"
@@ -134,10 +134,10 @@ digraph G {
     "unhealthy_leader_race" [label="Try to create leader key"]
     "unhealthy_leader_race" -> "unhealthy_acquire_lock"
     "unhealthy_acquire_lock" [label="Was I able to get the lock?", shape="diamond"]
-    "unhealthy_is_leader" [label="Is Postgres running as master?", shape=diamond]
+    "unhealthy_is_leader" [label="Is Postgres running as primary?", shape=diamond]
     "unhealthy_acquire_lock" -> "unhealthy_is_leader" [label="yes" color="green"]
     "unhealthy_is_leader" -> "unhealthy_promote" [label="no" color="red"]
-    "unhealthy_promote" [label="Promote to master"]
+    "unhealthy_promote" [label="Promote to primary"]
     "unhealthy_is_healthiest" -> "unhealthy_follow" [label="no" color="red"]
     "unhealthy_follow" [label="try to follow somebody else()"]
     "unhealthy_acquire_lock" -> "unhealthy_follow" [label="no" color="red"]

docs/ha_loop_diagram.png

@@ -37,7 +37,7 @@ Patroni Kubernetes :ref:`settings <kubernetes_settings>` and :ref:`environment v
 Customize role label
 ^^^^^^^^^^^^^^^^^^^^
 
-By default, Patroni will set corresponding labels on the pod it runs in based on node's role, such as ``role=master``.
+By default, Patroni will set corresponding labels on the pod it runs in based on node's role, such as ``role=primary``.
 The key and value of label can be customized by `kubernetes.role_label`, `kubernetes.leader_label_value`, `kubernetes.follower_label_value` and `kubernetes.standby_leader_label_value`.
 
 Note that if you migrate from default role labels to custom ones, you can reduce downtime by following migration steps:
@@ -48,16 +48,16 @@ Note that if you migrate from default role labels to custom ones, you can reduce
 
     labels:
       cluster-name: foo
-      role: master
-      tmp_role: master
+      role: primary
+      tmp_role: primary
 
 2. After all pods have been updated, modify the service selector to select the temporary label.
 
   .. code:: YAML
 
     selector:
       cluster-name: foo
-      tmp_role: master
+      tmp_role: primary
 
 3. Add your custom role label (e.g., set `kubernetes.leader_label_value=primary`). Once pods are restarted they will get following new labels set by Patroni:
 
@@ -66,7 +66,7 @@ Note that if you migrate from default role labels to custom ones, you can reduce
     labels:
       cluster-name: foo
       role: primary
-      tmp_role: master
+      tmp_role: primary
 
 4. After all pods have been updated again, modify the service selector to use new role value.
 

docs/kubernetes.rst

@@ -103,9 +103,9 @@ The ``GET /patroni`` is used by Patroni during the leader race. It also could be
     $ curl -s http://localhost:8008/patroni | jq .
     {
       "state": "running",
-      "postmaster_start_time": "2023-08-18 11:03:37.966359+00:00",
-      "role": "master",
-      "server_version": 150004,
+      "postmaster_start_time": "2024-08-18 11:03:37.966359+00:00",
+      "role": "primary",
+      "server_version": 160004,
       "xlog": {
         "location": 67395656
       },
@@ -134,7 +134,7 @@ The ``GET /patroni`` is used by Patroni during the leader race. It also could be
       },
       "database_system_identifier": "7268616322854375442",
       "patroni": {
-        "version": "3.1.0",
+        "version": "4.0.0",
         "scope": "demo",
         "name": "patroni1"
       }
@@ -147,9 +147,9 @@ The ``GET /patroni`` is used by Patroni during the leader race. It also could be
     $ curl -s http://localhost:8008/patroni  | jq .
     {
       "state": "running",
-      "postmaster_start_time": "2023-08-18 11:09:08.615242+00:00",
+      "postmaster_start_time": "2024-08-18 11:09:08.615242+00:00",
       "role": "replica",
-      "server_version": 150004,
+      "server_version": 160004,
       "xlog": {
         "received_location": 67419744,
         "replayed_location": 67419744,
@@ -182,7 +182,7 @@ The ``GET /patroni`` is used by Patroni during the leader race. It also could be
       },
       "database_system_identifier": "7268616322854375442",
       "patroni": {
-        "version": "3.1.0",
+        "version": "4.0.0",
         "scope": "demo",
         "name": "patroni1"
       }
@@ -195,9 +195,9 @@ The ``GET /patroni`` is used by Patroni during the leader race. It also could be
     $ curl -s http://localhost:8008/patroni  | jq .
     {
       "state": "running",
-      "postmaster_start_time": "2023-08-18 11:09:08.615242+00:00",
+      "postmaster_start_time": "2024-08-18 11:09:08.615242+00:00",
       "role": "replica",
-      "server_version": 150004,
+      "server_version": 160004,
       "xlog": {
         "location": 67420024
       },
@@ -228,7 +228,7 @@ The ``GET /patroni`` is used by Patroni during the leader race. It also could be
       },
       "database_system_identifier": "7268616322854375442",
       "patroni": {
-        "version": "3.1.0",
+        "version": "4.0.0",
         "scope": "demo",
         "name": "patroni1"
       }
@@ -241,9 +241,9 @@ The ``GET /patroni`` is used by Patroni during the leader race. It also could be
     $ curl -s http://localhost:8008/patroni  | jq .
     {
       "state": "running",
-      "postmaster_start_time": "2023-08-18 11:09:08.615242+00:00",
+      "postmaster_start_time": "2024-08-18 11:09:08.615242+00:00",
       "role": "replica",
-      "server_version": 150004,
+      "server_version": 160004,
       "xlog": {
         "location": 67420024
       },
@@ -273,7 +273,7 @@ The ``GET /patroni`` is used by Patroni during the leader race. It also could be
       },
       "database_system_identifier": "7268616322854375442",
       "patroni": {
-        "version": "3.1.0",
+        "version": "4.0.0",
         "scope": "demo",
         "name": "patroni1"
       }
@@ -287,16 +287,13 @@ Retrieve the Patroni metrics in Prometheus format through the ``GET /metrics`` e
 	
 	# HELP patroni_version Patroni semver without periods. \
 	# TYPE patroni_version gauge
-	patroni_version{scope="batman",name="patroni1"} 020103
+	patroni_version{scope="batman",name="patroni1"} 040000
 	# HELP patroni_postgres_running Value is 1 if Postgres is running, 0 otherwise.
 	# TYPE patroni_postgres_running gauge
 	patroni_postgres_running{scope="batman",name="patroni1"} 1
 	# HELP patroni_postmaster_start_time Epoch seconds since Postgres started.
 	# TYPE patroni_postmaster_start_time gauge
 	patroni_postmaster_start_time{scope="batman",name="patroni1"} 1657656955.179243
-	# HELP patroni_master Value is 1 if this node is the leader, 0 otherwise.
-	# TYPE patroni_master gauge
-	patroni_master{scope="batman",name="patroni1"} 1
 	# HELP patroni_primary Value is 1 if this node is the leader, 0 otherwise.
 	# TYPE patroni_primary gauge
 	patroni_primary{scope="batman",name="patroni1"} 1
@@ -335,7 +332,7 @@ Retrieve the Patroni metrics in Prometheus format through the ``GET /metrics`` e
 	patroni_postgres_in_archive_recovery{scope="batman",name="patroni1"} 0
 	# HELP patroni_postgres_server_version Version of Postgres (if running), 0 otherwise.
 	# TYPE patroni_postgres_server_version gauge
-	patroni_postgres_server_version{scope="batman",name="patroni1"} 140004
+	patroni_postgres_server_version{scope="batman",name="patroni1"} 160004
 	# HELP patroni_cluster_unlocked Value is 1 if the cluster is unlocked, 0 if locked.
 	# TYPE patroni_cluster_unlocked gauge
 	patroni_cluster_unlocked{scope="batman",name="patroni1"} 0
@@ -497,18 +494,18 @@ Let's check that the node processed this configuration. First of all it should s
 	{
 	  "pending_restart": true,
 	  "database_system_identifier": "6287881213849985952",
-	  "postmaster_start_time": "2016-06-13 13:13:05.211 CEST",
+	  "postmaster_start_time": "2024-08-18 13:13:05.211 CEST",
 	  "xlog": {
 	    "location": 2197818976
 	  },
 	  "patroni": {
-	    "version": "1.0",
+	    "version": "4.0.0",
 	    "scope": "batman",
 	    "name": "patroni1"
 	  },
 	  "state": "running",
-	  "role": "master",
-	  "server_version": 90503
+	  "role": "primary",
+	  "server_version": 160004
 	}
 
 Removing parameters:

docs/rest_api.rst

@@ -104,7 +104,7 @@ Most of the parameters are optional, but you have to specify one of the **host**
 -  **consistency**: (optional) Select consul consistency mode. Possible values are ``default``, ``consistent``, or ``stale`` (more details in `consul API reference <https://www.consul.io/api/features/consistency.html/>`__)
 -  **checks**: (optional) list of Consul health checks used for the session. By default an empty list is used.
 -  **register\_service**: (optional) whether or not to register a service with the name defined by the scope parameter and the tag master, primary, replica, or standby-leader depending on the node's role. Defaults to **false**.
--  **service\_tags**: (optional) additional static tags to add to the Consul service apart from the role (``master``/``primary``/``replica``/``standby-leader``). By default an empty list is used.
+-  **service\_tags**: (optional) additional static tags to add to the Consul service apart from the role (``primary``/``replica``/``standby-leader``). By default an empty list is used.
 -  **service\_check\_interval**: (optional) how often to perform health check against registered url. Defaults to '5s'.
 -  **service\_check\_tls\_server\_name**: (optional) overide SNI host when connecting via TLS, see also `consul agent check API reference <https://www.consul.io/api-docs/agent/check#tlsservername>`__.
 
@@ -178,11 +178,11 @@ Kubernetes
 -  **namespace**: (optional) Kubernetes namespace where Patroni pod is running. Default value is `default`.
 -  **labels**: Labels in format ``{label1: value1, label2: value2}``. These labels will be used to find existing objects (Pods and either Endpoints or ConfigMaps) associated with the current cluster. Also Patroni will set them on every object (Endpoint or ConfigMap) it creates.
 -  **scope\_label**: (optional) name of the label containing cluster name. Default value is `cluster-name`.
--  **role\_label**: (optional) name of the label containing role (master or replica or other custom value). Patroni will set this label on the pod it runs in. Default value is ``role``.
--  **leader\_label\_value**: (optional) value of the pod label when Postgres role is ``master``. Default value is ``master``.
+-  **role\_label**: (optional) name of the label containing role (`primary`, `replica`, or other custom value). Patroni will set this label on the pod it runs in. Default value is ``role``.
+-  **leader\_label\_value**: (optional) value of the pod label when Postgres role is ``primary``. Default value is ``primary``.
 -  **follower\_label\_value**: (optional) value of the pod label when Postgres role is ``replica``. Default value is ``replica``.
--  **standby\_leader\_label\_value**: (optional) value of the pod label when Postgres role is ``standby_leader``. Default value is ``master``.
--  **tmp_\role\_label**: (optional) name of the temporary label containing role (master or replica). Value of this label will always use the default of corresponding role. Set only when necessary.
+-  **standby\_leader\_label\_value**: (optional) value of the pod label when Postgres role is ``standby_leader``. Default value is ``primary``.
+-  **tmp_\role\_label**: (optional) name of the temporary label containing role (`primary` or `replica`). Value of this label will always use the default of corresponding role. Set only when necessary.
 -  **use\_endpoints**: (optional) if set to true, Patroni will use Endpoints instead of ConfigMaps to run leader elections and keep cluster state.
 -  **pod\_ip**: (optional) IP address of the pod Patroni is running in. This value is required when `use_endpoints` is enabled and is used to populate the leader endpoint subsets when the pod's PostgreSQL is promoted.
 -  **ports**: (optional) if the Service object has the name for the port, the same name must appear in the Endpoint object, otherwise service won't work. For example, if your service is defined as ``{Kind: Service, spec: {ports: [{name: postgresql, port: 5432, targetPort: 5432}]}}``, then you have to set ``kubernetes.ports: [{"name": "postgresql", "port": 5432}]`` and Patroni will use it for updating subsets of the leader Endpoint. This parameter is used only if `kubernetes.use_endpoints` is set.

docs/yaml_configuration.rst

@@ -65,7 +65,7 @@ Feature: basic replication
     Then I receive a response returncode 0
     And postgres2 role is the primary after 24 seconds
     And Response on GET http://127.0.0.1:8010/history contains recovery after 10 seconds
-    And there is a postgres2_cb.log with "on_role_change master batman" in postgres2 data directory
+    And there is a postgres2_cb.log with "on_role_change primary batman" in postgres2 data directory
     When I issue a PATCH request to http://127.0.0.1:8010/config with {"synchronous_mode": null, "master_start_timeout": 0}
     Then I receive a response code 200
     When I add the table bar to postgres2

features/basic_replication.feature

@@ -69,7 +69,7 @@ Feature: citus
   Scenario: check that in-flight transaction is rolled back after timeout when other workers need to change pg_dist_node
     Given I start postgres4 in citus group 2
     Then postgres4 is a leader in a group 2 after 10 seconds
-    And "members/postgres4" key in a group 2 in DCS has role=master after 3 seconds
+    And "members/postgres4" key in a group 2 in DCS has role=primary after 3 seconds
     When I run patronictl.py edit-config batman --group 2 -s ttl=20 --force
     Then I receive a response returncode 0
     And I receive a response output "+ttl: 20"