Keep synchronizing slots when others are lagging on primary.

[rdunklau]

Instead of blocking indefinitely for a replication slot to be syncable, introduce a new GUC pg_failover_slots.sync_timeout after which we will move to the next one.

To avoid waiting from scratch, we create the replication as temporary ones instead of ephemeral ones, allowing them to keep their state between runs. When the slot is finally synced, we persist it to disk.

Since we do not block in waiting state anymore, we need to cleanup the inconsistent slots after promotion.

This solves the possible issue of having an inactive slot in the primary which prevents every other slots to be synced to the standby.

[kfcss]

We have noticed the same problems. This results in the secondary consuming more disk space than the primary.

[PJMODOS]

This looks interesting, I am starting to wonder whether we need some kind of status function that will report state of all the slots in some user consumable way.

[rdunklau]

This looks interesting, I am starting to wonder whether we need some kind of status function that will report state of all the slots in some user consumable way.

For monitoring purposes that's a pretty good idea. As of now, one can rely on the persistence of the slot to infer it's status but it's not ideal.

[rdunklau]

Do you have any opinion on that design ?

[rdunklau]

I just rebased it on the current master branch.

[jmealo]

@petere @PJMODOS Curious if this will get merged? 🤔