Introduce `experimental-config-options` configuration property

[mkleczek]

Problem

Decision to introduce a new configuration option is always difficult. Several factors have to be taken into account, such as:

• semantics of the configuration option (ie. opt-in vs opt-out, boolean flag vs numeric)
• naming of the configuration option
• optionality
• default value
  etc.

Ideally we should try to avoid introducing new configuration options if possible and provide software behaving "properly" without requiring any input from administrator/user. What "proper" means though is very often not obvious and depends on field experience.

Discussion in #4643 (comment) illustrates this pretty well: without the actual information from the field it is difficult to decide if we should introduce schema loading throttling configuration at all (and how it should look like).

Another good use case is feature flags. The discussion in #4236 (comment) is a good illustration of the need for feature flags. Adding enable_self_links configuration property (defaulting to false) is a problem right now because of future backward compatibility concerns. Gating it as an experimental config option would allow us to experiment with self links and gather information about the needs of the users.

Solution

Introduce a concept of experimental config options and experimental-config-options configuration option (yeah, I know :) ) containing an explicit list of experimental option names used in configuration.
Experimental config option can appear and disappear at any time (even in minor releases). Providing of experimental config option value would give an error if the option's name is not provided in experimental-config-options property.

The idea is to allow experimenting with new configuration options making sure users consciously and explicitly opt-in to configure them. That way we don't have to pay that much attention to long term backward compatibility when trying new config options. It should also allow us to gather information about feasibility of the options: removal of widely used experimental option would cause feedback (ie. Github issues) that we could treat as a signal to make such option non-experimental.

[wolfgangwalther]

Ideally we should try to avoid introducing new configuration options if possible and provide software behaving "properly" without requiring any input from administrator/user.

Very much agree here!

What "proper" means though is very often not obvious and depends on field experience.

I think that's true for stuff that needs to be "tuned" to the specific environment. However, some things can probably use different / smart algorithms, which are "auto-tuning", in a way. Example: Instead of configuring retry-delays, often enough exponential backoff is used instead. That's tuning itself during execution.

Imho, there often are cases where the ability to set something via configuration only comes up, because a sub-optimal solution is implemented in the first place.

Discussion in #4643 (comment) illustrates this pretty well: without the actual information from the field it is difficult to decide if we should introduce schema loading throttling configuration at all (and how it should look like).

See #4643 (comment) for my opinion on that.

Another good use case is feature flags. The discussion in #4236 (comment) is a good illustration of the need for feature flags. Adding enable_self_links configuration property (defaulting to false) is a problem right now because of future backward compatibility concerns. Gating it as an experimental config option would allow us to experiment with self links and gather information about the needs of the users.

Our concept of "feature flags", so far, has been the ability to just run PostgREST from any PR and test it that way. We upload the built artifacts in each GHA run, so that people can use them. Not sure whether we want to start releasing experimental features.

[mkleczek]

Ideally we should try to avoid introducing new configuration options if possible and provide software behaving "properly" without requiring any input from administrator/user.

Very much agree here!

What "proper" means though is very often not obvious and depends on field experience.

I think that's true for stuff that needs to be "tuned" to the specific environment. However, some things can probably use different / smart algorithms, which are "auto-tuning", in a way. Example: Instead of configuring retry-delays, often enough exponential backoff is used instead. That's tuning itself during execution.

Imho, there often are cases where the ability to set something via configuration only comes up, because a sub-optimal solution is implemented in the first place.

That's very often true, indeed. We all dream about "self-configuring" and "self-tuning" software, and we should strive to achieve that.
In practice configuration is just unavoidable.

Discussion in #4643 (comment) illustrates this pretty well: without the actual information from the field it is difficult to decide if we should introduce schema loading throttling configuration at all (and how it should look like).

See #4643 (comment) for my opinion on that.

IMHO this opinion just proves the point: coming up with the right default is difficult. The idea is then to:

• provide a default
• provide an experimental config option to enable overriding the default
• wait and see if the experimental property is used in the field: if it is then we need to reconsider the default or make the property non-experimental, if it's not - we can get rid of it.

Another good use case is feature flags. The discussion in #4236 (comment) is a good illustration of the need for feature flags. Adding enable_self_links configuration property (defaulting to false) is a problem right now because of future backward compatibility concerns. Gating it as an experimental config option would allow us to experiment with self links and gather information about the needs of the users.

Our concept of "feature flags", so far, has been the ability to just run PostgREST from any PR and test it that way. We upload the built artifacts in each GHA run, so that people can use them.

This is very problematic to use in any professional environment as the PR build does not receive bug fixes etc. So it puts operational burden on users who now have to set-up the whole Nix/Haskell stack and know-how to even build the project (not even mentioning patching it). So in practice nobody uses PR builds and we are stuck with handwaving about need (or the lack of) for certain features.