Configuration Merging for Developer Experience

[abhisek]

When I update PMG and want to use sandbox after #101 I am restricted.

• I cannot just edit config because my config template is old
• I cannot just delete config and re-create because I have custom changes in my config with comments

But after I update, I want to start using the feature. I have a few option:

• Manually see the config template (docs) and update my config file
• Enable sandbox in the config file

This issue is to explore if a better dev-ex is possible with config handling. Especially when PMG is updated and user run pmg setup install, is it possible to incrementally update config with our default values?

Alternatively, ship PMG with default config which can be overriden in config.yml.

[Sahilb315]

@abhisek

Faced the same issue.

We could also have a flag --override-config, if user wants to explicitly override the config with new config.

Alternatively, ship PMG with default config which can be overridden in config.yml.

It's good as an temporary fix but not future proof. Consider a feature not having a flag for it to be enabled and can only be enabled through config, the user would have to manually update the config by seeing the config template

I think incremental update (merging existing and new config options) or overriding the config seem better options.

[abhisek]

I am a bit against flags like --override-config because I don't know how to use it. For me, PMG has 2 flows:

1. Installation & setup (rare)
2. Usage

I am happy to run few commands during [1]. But when I am in my dev loop, I don't want to know that PMG exist. So I can't really think of adding a arg.

I think incremental update (merging existing and new config options) or overriding the config seem better options.

How can we do this transparently during pmg setup install? Also in this approach, we need to document that pmg setup install must be run after every update. We also need to guarantee that pmg setup install is non-breaking and only extends the configuration without changing existing customizations.

Another alternative is to allow config / sandbox policy override from .safedep/pmg-config.yml and .safedep/pmg-sandbox-policy.yml. These are per project (repository) config and policy overrides. I am not sure about config, still feel like config should be a single file (single source of truth) for a given system (machine).

But most likely we will need per-project sandbox policy overrides due to the nature of least privilege sandboxing.

[Sahilb315]

How can we do this transparently during pmg setup install? Also in this approach, we need to document that pmg setup install must be run after every update. We also need to guarantee that pmg setup install is non-breaking and only extends the configuration without changing existing customizations.

I was thinking if we can update the config dynamically.
Example scenario:
pmg setup install -> Creates config.yml -> PMG gets a config update -> User pulls the latest/updated version -> User continues it existing flow -> pmg npm i ... -> Check for updated config template and existing user config

But it would add a overhead of checking for config on each command execution.

Another alternative is to allow config / sandbox policy override from .safedep/pmg-config.yml and .safedep/pmg-sandbox-policy.yml. These are per project (repository) config and policy overrides. I am not sure about config, still feel like config should be a single file (single source of truth) for a given system (machine).
But most likely we will need per-project sandbox policy overrides due to the nature of least privilege sandboxing.

Yeah, I agree. Only sandbox policy would be needed (if required) per-project, introducing pmg config per-project would be a bit over and shouldn't be required.

@abhisek What about introducing a new command pmg upgrade, which updates the pmg version and we can even do config updates and any other required modifications in future. User wouldn't have to separately execute a upgrade cmd (brew upgrade safedep/tap/pmg) and then pmg setup install. This command would be combination of both and easier for user to update

[abhisek]

@Sahilb315 We should not do upgrade. Eventually PMG should be packaged as a OS specific package such as deb, yum and other package managers like pacman. They have their own best practices when it comes to packaging, update etc.

We should not reinvent the wheel in the long term.

For short term, we probably need:

• Simple update check and inform user if a new version is available without friction

Let package managers (like homebrew and in future other package managers, take care of the upgrade flow). I think Homebrew can be configured to run install hooks like pmg setup install. But then what?

What should be the behaviour of pmg setup install when it comes to an existing config file.

One option I can think of, which I have seen in Omarchy, Rails and Gentoo is, show a diff to user, on user acceptance, apply the diff. This means:

• PMG has its own updated config template
• During setup install, it computes diff between what is available in system and its latest config template
• If there is a change, show that to user, ask if the template changes should be applied to their current config
• On user conifmration, backup existing config, apply changes (confirmed by user) to the config

What do you think of this approach?

[Sahilb315]

@abhisek
If we require user to run pmg setup install, how hard would it be to merge the existing (user) config and PMG updated config template i.e simply adding any new/missing keys while preserving existing values? Isn't is less friction for user and much easier?

Computing a diff and asking for confirmation feels over-engineered for our case.

[anantshri]

One way could be how https://codeberg.org/superseriousbusiness/gotosocial/releases these people manage releases. for each release there is a diff option on sample config file to see what changed and they leave it up to the user to modify config or update config.