Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
a7d7466
Implement multisite-switchover CTL command
Mar 26, 2025
2be30da
Solve a small logic issue
Mar 27, 2025
12e0df5
Set candidate name reliably if there is only one
Mar 27, 2025
f772966
Fix some type hinting and import issues
Apr 17, 2025
412d0a6
Address further type checking issues
Apr 17, 2025
0804b93
Simplify site switchover name
Apr 18, 2025
ec9b6c3
Merge branch 'multisite' into ctl-switchover-only
May 12, 2025
78d8607
Address some flake8 complaints
May 13, 2025
d3d1e47
Limit py-consul version depending on python version (#3336)
CyberDem0n Apr 18, 2025
b47d7b9
Convert roles to enums (#3303)
hughcapet Apr 18, 2025
07b81cd
Convert states to enums (#3293)
hughcapet Mar 14, 2025
ed70e0e
Convert roles to enums (#3303)
hughcapet Apr 18, 2025
2001406
Adapt unit tests to multisite changes
May 13, 2025
6ed7554
Cherry-pivk some upstream changes
hughcapet Apr 18, 2025
bf5746e
Implement kubernetes.bootstrap_labels (#3257)
hughcapet Feb 18, 2025
d49c0ba
Solve a couple of Flaky unit tests (#3294)
Garaz08 Feb 25, 2025
ab43b49
Pick changes manually that were missed during cherry picking
May 14, 2025
2d10c10
Fix some linting issues
May 14, 2025
9f20479
Make sure str representation is used for click options (#3352)
hughcapet May 12, 2025
482e68c
Merge branch 'multisite' into ctl-switchover-only
ants May 15, 2025
3bf317a
Fix a small isort issue
May 15, 2025
db359d6
Fix most Pyright issues in multisite.py
May 15, 2025
ce73bff
Fix most Pyright issues in ha.py
May 15, 2025
b1f204e
Fix method call
May 16, 2025
94fbf0b
Fix remaining pyright issues that can be fixed without implementation…
May 19, 2025
49f8c25
Change MS history to the format of normal history
May 21, 2025
84c351d
Make argument which we do not use optional
May 21, 2025
a5a9c39
Fix tests failing with newest click
May 22, 2025
50c9785
Fix some RST
May 23, 2025
aed5381
Add multisite.rst to TOC
May 23, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ Currently supported PostgreSQL versions: 9.3 to 17.
tools_integration
security
ha_multi_dc
multisite
faq
releases
CONTRIBUTING
Expand Down
117 changes: 59 additions & 58 deletions docs/multisite.rst
Original file line number Diff line number Diff line change
Expand Up @@ -91,48 +91,49 @@ The configuration is very similar to the usual Patroni config. In fact, the key

An example configuration for two Patroni sites:

```
multisite:
name: dc1
namespace: /multisite/
etcd3: # <DCS>
hosts:
# dc1
- 10.0.1.1:2379
- 10.0.1.2:2379
- 10.0.1.3:2379
# dc2
- 10.0.2.1:2379
- 10.0.2.2:2379
- 10.0.2.3:2379
# dc 3
- 10.0.0.1:2379
host: 10.0.1.1,10.0.1.2,10.0.1.3 # How the leader of the other site(s) can connect to the primary on this site
port: 5432
# Multisite failover timeouts
ttl: 90
retry_timeout: 40
```
.. code:: YAML

multisite:
name: dc1
namespace: /multisite/
etcd3: # <DCS>
hosts:
# dc1
- 10.0.1.1:2379
- 10.0.1.2:2379
- 10.0.1.3:2379
# dc2
- 10.0.2.1:2379
- 10.0.2.2:2379
- 10.0.2.3:2379
# dc 3
- 10.0.0.1:2379
host: 10.0.1.1,10.0.1.2,10.0.1.3 # How the leader of the other site(s) can connect to the primary on this site
port: 5432
# Multisite failover timeouts
ttl: 90
retry_timeout: 40


Details of the configuration parameters
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

`name`
: The name of the site. All nodes that share the same value are considered to be a part of the same site, thus it must be different for each site.
`namespace`
: Optional path within DCS where Patroni stores the multisite state. If used, it should be different from the namespace used by the base config, but the same on all sites.
`<DCS>` (in the example `etcd3`)
: The DCS implementation in use. Possible values are `etcd`, `etcd3`, `zookeeper`, `consul`, `exhibitor`, `kubernetes`, or `raft` (the latter is deprecated).
`<DCS>.hosts`
: a list of IP addresses of nodes forming the global DCS cluster, including the extra (tiebreaking) node(s)
`host`
: Comma-separated list of IPs of the Patroni nodes that can become a primary on the present site
`port`
: Postgres port, through which other sites' members can connect to this site. It can be specified once if all nodes use the same port, or as a comma-separated list matching the different port numbers, in the order used in the `host` key.
`ttl`
: Time to live of site leader lock. If the site is unable to elect a functioning leader within this timeout, a different site can take over the leader role. Must be a few times longer than the usual `ttl` value in order to prevent unnecessary site failovers.
`retry_timeout`
: How long the global etcd cluster can be inaccessible before the cluster is demoted. Must be a few times longer than the usual `retry_timeout` value in order to prevent unnecessary site failovers.
``name``
The name of the site. All nodes that share the same value are considered to be a part of the same site, thus it must be different for each site.
``namespace``
Optional path within DCS where Patroni stores the multisite state. If used, it should be different from the namespace used by the base config, but the same on all sites.
``<DCS>`` (in the example ``etcd3``)
The DCS implementation in use. Possible values are ``etcd``, ``etcd3``, ``zookeeper``, ``consul``, ``exhibitor``, ``kubernetes``, or ``raft`` (the latter is deprecated).
``<DCS>.hosts``
a list of IP addresses of nodes forming the global DCS cluster, including the extra (tiebreaking) node(s)
``host``
Comma-separated list of IPs of the Patroni nodes that can become a primary on the present site
``port``
Postgres port, through which other sites' members can connect to this site. It can be specified once if all nodes use the same port, or as a comma-separated list matching the different port numbers, in the order used in the ``host`` key.
``ttl``
Time to live of site leader lock. If the site is unable to elect a functioning leader within this timeout, a different site can take over the leader role. Must be a few times longer than the usual ``ttl`` value in order to prevent unnecessary site failovers.
``retry_timeout``
How long the global etcd cluster can be inaccessible before the cluster is demoted. Must be a few times longer than the usual ``retry_timeout`` value in order to prevent unnecessary site failovers.

Passwords in the YAML configuration
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Expand Down Expand Up @@ -184,23 +185,23 @@ Applications should be ready to try to connect to the new primary. See 'Connect
Glossary
++++++++

DCS
: distributed configuration store
site
: a Patroni cluster with any number of nodes, and the respective DCS - usually corresponding to a data centre
primary
: the writable PostgreSQL node, from which the other nodes replicate their data (either directly or in a cascading fashion)
leader
: the node which other nodes inside the same site replicate from - the leader can be a replica itself, in which case it's called a _standby leader_
site switchover
: a (manual) leader site switch performed when both sites are functioning fine
site failover
: when the main site goes down (meaning there is no Patroni leader and none of the remaining nodes (if any left) can become a leader), the standby leader will be promoted, becoming a leader proper, and the Postgres instance running there becoming the primary
leader site
: the site where the PostgreSQL primary instance is
standby site
: a site replicating from the leader site, and a potential target for site switchover/failover
DCS quorum
: more than half of the DCS nodes are available (and can take part in a leader race)
multisite leader lock
: just like under normal Patroni operation, the leader puts/updates an entry in DCS, thus notifying other sites that there is a functioning Postgres primary running. The entry mentioned is the multisite leader lock.
**DCS**
distributed configuration store
**site**
a Patroni cluster with any number of nodes, and the respective DCS - usually corresponding to a data centre
**primary**
the writable PostgreSQL node, from which the other nodes replicate their data (either directly or in a cascading fashion)
**leader**
the node which other nodes inside the same site replicate from - the leader can be a replica itself, in which case it's called a *standby leader*
**site switchover**
a (manual) leader site switch performed when both sites are functioning fine
**site failover**
when the main site goes down (meaning there is no Patroni leader and none of the remaining nodes (if any left) can become a leader), the standby leader will be promoted, becoming a leader proper, and the Postgres instance running there becoming the primary
**leader site**
the site where the PostgreSQL primary instance is
**standby site**
a site replicating from the leader site, and a potential target for site switchover/failover
**DCS quorum**
more than half of the DCS nodes are available (and can take part in a leader race)
**multisite leader lock**
just like under normal Patroni operation, the leader puts/updates an entry in DCS, thus notifying other sites that there is a functioning Postgres primary running. The entry mentioned is the multisite leader lock.
8 changes: 4 additions & 4 deletions patroni/api.py
Original file line number Diff line number Diff line change
Expand Up @@ -681,9 +681,9 @@ def do_GET_metrics(self) -> None:
metrics.append("# HELP patroni_multisite_switches Number of times multisite leader has been switched")
metrics.append("# TYPE patroni_multisite_switches counter")
metrics.append("patroni_multisite_switches{0} {1}"
.format(labels, patroni.multisite.site_switches))
.format(labels, patroni.multisite.site_switches)) # noqa: E501 # pyright: ignore [reportUnknownMemberType, reportUnknownArgumentType, reportAttributeAccessIssue]

self.write_response(200, '\n'.join(metrics)+'\n', content_type='text/plain')
self.write_response(200, '\n'.join(metrics) + '\n', content_type='text/plain')

def do_GET_multisite(self):
self._write_json_response(200, self.server.patroni.multisite.status())
Expand Down Expand Up @@ -1199,13 +1199,13 @@ def do_POST_switchover(self) -> None:
self.do_POST_failover(action='switchover')

@check_access
def do_POST_multisite_switchover(self):
def do_POST_site_switchover(self) -> None:
request = self._read_json_content()
(status_code, data) = (400, '')
if not request:
return
if not self.server.patroni.multisite.is_active:
return self._write_response(400, 'Cluster is not in multisite mode')
return self.write_response(400, 'Cluster is not in multisite mode')

scheduled_at = request.get('scheduled_at')
target_site = request.get('target_site')
Expand Down
Loading
Loading