Add env lock command to generate PEP compliant lock files.

[cjames23]

This is a draft to start discussions around the design here for PEP 751 support by creating per environment lockfiles.

As of now this would currently only support a user triggering the lock through hatch env lock MyEnvName or hatch env lock --all

I would like to get inputs here on if we want to support auto lockfile creation on environment creation or not?

[flying-sheep]

Generally this looks good as a first step, but we need to think about the way forward. I think the default way to use lockfiles with hatch should be declarative like the rest of env management.

First of all, there is one issue: pylock.hatch-test.py3.14.lock.toml isn’t a spec-compliant file name! The spec only allows one dot in the filename. But read on.

I think imperatively updating a lockfile like this PR enables is mosty useful for the --upgrade[-all] flags here, so I think the -o/--output flag should maybe be called --export instead, so people can use it to create lockfiles for envs that aren’t configured to have lockfiles (see below). I’d also say that ideally, --export should be mandatory for now, to make clear that hatch doesn’t use that lockfile for anything.

This would facilitate a design like the following:

design

Personally, I think I’d like to see

• declare in the config which environments are locked, maybe using a combination of a global lock-envs = true | false setting, which can be overridden per environment (using standard environment settings compatible with override)¹
• hatch env create and hatch run <some-locked-env>:<command> would
  • locally: sync or create the lockfile
  • in CI: use --locked by default, raising an error if the file isn’t up-to-date
• hatch lock would be mainly used to update individual dependencies in lockfiles and things like that (as I said in the first sentence of this comment)

reusing lockfiles

We should integrate with dependency-groups and extras in the lockfile. The spec says that these fields mean that the lockfile “supports” these dependency groups and extras. The idea for these fields is that a lockfile is reusable for multiple environments with different combinations of extras/dependency groups.

I’d like to be able to define e.g. a features matrix variable, and then have the same lockfile for all hatch-test envs with them using uv pip sync --group=... --extra=... pylock.hatch-test.toml

all together:

By default, the following config would make hatch test ... sync a pylock.hatch-test.toml file, which would contain dependency-groups = [ "test" ] and extras = [ "feat1" ]

In each of the respective environments,

• hatch test -i extras=min would call uv pip sync --group=test pylock.hatch-test.toml
• hatch test -i extras=full would call uv pip sync --group=test --extra=feat1 pylock.hatch-test.toml

Note

if overrides.*.dependencies is used, that behavior would not be possible, so we’d have to disallow locking environments that use dependencies overrides.²

[envs.hatch-test]
locked = true
dependency-groups = [ "test" ]
overrides.matrix.extras.features = [
    { if = [ "all" ], value = "feat1" },
]

[[envs.hatch-test.matrix]]
python = [ "3.14", "3.10" ]
extras = [ "none", "all" ]

Running hatch lock hatch-test would then create/sync the lockfile and all hatch-test envs, hatch lock would continue error without --export as the default env isn’t configured to be locked.

not considered

I used uv pip in the examples, and pip should work the same. I don’t know if we want to support uv.lock, since it might have different paradigms regarding the lockfile reuse above and might make things more complicated.

Footnotes

1. too bad hatch-test doesn’t inherit from the default env, otherwise we wouldn’t need a global setting and could just set the “locked” setting on the default env. ↩

2. or do some weird name mangling to allow putting more matrix variables into the lockfile name, since only two dots are allowed. ↩

[cjames23]

I agree with the direction to be more per environment so I will start working on getting that implemented. I think the one thing that we might want to discuss is if it makes sense to lock internal environments by default and if internal environments should be allowed to be overridden for locking or not.

[flying-sheep]

What happens with using hatch env create or hatch run on an env that’s locked? Will the lockfile not be generated?

What happens when using hatch lock on an env that’s not locked? Will the same lockfile be generated as if it was locked? I think that’d lead to confusion. I think --export should be mandatory for envs that aren’t locked (with an error message that mention lock-envs and locked).

[flying-sheep]

what happens when some envs aren’t locked using the config? Will this also generate files for them that hatch then ignores? I think that would lead to much confusion.

Because the default behavior is

When no explicit environment name is passed, hatch env lock will lock all environments that have locked = true

I think either

• --all should just be removed and if --export=some/path/ ends in a /, lockfiles for all envs should be put into that directory.
• or (more explicit) --all should be renamed to --export-all=<PATH>, which is mutually exclusive with --export and writes lockfiles for all envs to the specified directory

[flying-sheep]

see above

[flying-sheep]

Hmm, this kind of name mangling is probably the best we can do.

[flying-sheep]

If multiple envs share one lock-filename, they need to be locked together.

E.g. if I do

[envs.hatch-test]
locked = true
overrides.matrix.deps.env-vars = [
  { if = [ 'pre' ], key = 'UV_PRERELEASE', value = 'allow' },
]
overrides.matrix.deps.lock-filename = [
  { if = [ 'stable' ], value = 'pylock.test-stable.lock' },
  { if = [ 'pre' ], value = 'pylock.test-pre.lock' },
]
overrides.matrix.extras.features = [
  { if = [ 'all' ], value = 'feat1' },
]

[[envs.hatch-test.matrix]]
python = [ '3.14', '3.10' ]
deps = [ 'stable', 'pre' ]
extras = [ 'none', 'all' ]

We want to see

pylock.test-stable.lock
pylock.test-pre.lock

And we want hatch to call uv pip compile exactly once per lockfile, with the combined dependencies, extras, and dependency-group of all environments.

As currently implemented, hatch would call it 3 times per environment, with it being a gamble which one wins.