[RFC 0197] package sets by-name #197
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,151 @@ | ||
| --- | ||
| feature: nixpkgs package set location | ||
| start-date: 2026-01-24 | ||
| author: quantenzitrone | ||
| co-authors: (find a buddy later to help out with the RFC) | ||
| shepherd-team: (names, to be nominated and accepted by RFC steering committee) | ||
| shepherd-leader: (name to be appointed by RFC steering committee) | ||
| related-issues: (will contain links to implementation PRs) | ||
| --- | ||
|
|
||
| # Summary | ||
| [summary]: #summary | ||
|
|
||
| Different ideas on how to handle package sets in nixpkgs. | ||
|
|
||
| # Motivation | ||
| [motivation]: #motivation | ||
|
|
||
| - get rid of the package categories as directories. | ||
quantenzitrone marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| - make packages in package sets take advantage of the by-name greatness | ||
| - auto updates with r-ryantm + merge bot maintainer merging | ||
|
||
| - *your goal here* | ||
This comment was marked as off-topic.
Sorry, something went wrong. There was a problem hiding this comment. Updates across package sets are very different. I don't think this should be a goal of this RFC, but instead something that can be developed separately as an |
||
|
|
||
| # Idea 1: new directory for package sets | ||
|
|
||
| Proof-Of-Concept implementation in https://github.com/NixOS/nixpkgs/pull/482538 and https://github.com/NixOS/nixpkgs/pull/482543 | ||
|
|
||
| ## Detailed design | ||
|
|
||
| - Create a new directory under `pkgs/`, e.g. `pkgs/sets` that contains package sets. | ||
| - For example `pkgs/sets/fishPlugins` `pkgs/sets/python3Packages`. | ||
quantenzitrone marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| - Each packageset has a `packageset.nix`, that functions as an entrypoint to the package set. | ||
quantenzitrone marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| - The `packageset.nix` files are autocalled by some `pkgs/top-level/by-name-overlay.nix` like file. | ||
quantenzitrone marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| - Versioned package sets like `python316Packages` are done in `all-packages.nix` by calling the | ||
|
||
| `packageset.nix` again with different arguments. | ||
|
|
||
| ## Advantages | ||
|
|
||
| - flexibility | ||
| - easy transition | ||
| - for a lot of package sets just move the folder and rename the `default.nix` to `packageset.nix` | ||
|
|
||
quantenzitrone marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| ## Drawbacks | ||
|
|
||
| - all packagesets have to handle their shit themselves, so no maintainer merging, by-name | ||
| autocalling and stuff without the additional work | ||
quantenzitrone marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| - *your drawback here* | ||
|
|
||
| ## Unresolved Questions | ||
|
|
||
| - *your question here* | ||
quantenzitrone marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| # Idea 2: nested by-name structure | ||
|
|
||
| Proof-Of-Concept implementation in https://github.com/NixOS/nixpkgs/pull/483432 | ||
|
|
||
| ## Detailed design | ||
|
|
||
| - Package sets are done in a nested `by-name` structure under `pkgs/by-name`, e.g. | ||
| `fishPlugins.puffer` would be in `by-name/fi/fishPlugins/pu/puffer`. | ||
| - To distinguish packages from packagesets we have to mark packagesets somehow, for example with a | ||
| file `by-name/fi/fishPlugins/.packageset`, if that file exists, it's called as a packageset, | ||
| if not the `package.nix` must exist and is called as a package. | ||
quantenzitrone marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| ## Advantages | ||
|
|
||
| - clear package separation | ||
| - reuse of RFC 140 concepts | ||
| - making maintainer merging work for this is probably relatively simple | ||
|
|
||
| ## Drawbacks | ||
|
|
||
| - unresolved questions | ||
| - *your drawback here* | ||
quantenzitrone marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| ## Unresolved Questions | ||
|
|
||
| - How do we handle functions like `fishPlugins.buildFishPlugin`? | ||
| - How do we handle aliases? | ||
quantenzitrone marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| - How do we handle versioned package sets? | ||
| - How do we move large package sets? | ||
|
|
||
| # Idea 3: package sets in `pkgs/by-name` (probably worse than Idea 2 in every way) | ||
|
|
||
| Proof-Of-Concept implementation in https://github.com/NixOS/nixpkgs/pull/483128 | ||
|
|
||
| ## Detailed design | ||
|
|
||
| - Instead of `by-name/<shard>/<pname>` we have `by-name/<shard>/<attrpath>`, so e.g. | ||
| `fishPlugins.puffer` would go in `by-name/fi/fishPlugins.puffer`. | ||
| - The `top-level/by-name-overlay.nix` will call all folders in a `<shard>` that contain a dot as a | ||
| package set. | ||
|
|
||
| ## Advantages | ||
|
|
||
| - clear package separation | ||
| - reuse of RFC 140 concepts | ||
| - making maintainer merging work for this is probably easy | ||
|
|
||
| ## Drawbacks | ||
|
|
||
| - huge shards due to package sets | ||
| - currently only few shards like `li` are too large for GitHubs UI, but with this idea a lot of | ||
| shards like `py` (`python3Packages`), `ha` (`haskellPackages`) or `vi` (`vimPlugins`) will be | ||
quantenzitrone marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| huge as well. | ||
| - unresolved questions | ||
|
|
||
| ## Unresolved Questions | ||
|
|
||
| - How do we handle functions like `fishPlugins.buildFishPlugin`? | ||
|
There was a problem hiding this comment. (Answering because this applies to idea 1) I never thought it made sense for functions and derivation builders to be inside package sets. I think our Go infrastructure has the right idea: move them to the top-level, and version them in the attribute name.
Member
There was a problem hiding this comment. This proposal is awkward with Python where you have CPython and PyPy and then language version-dialects within them, and even worse for Common Lisp where you have same base language but multiple compatible implementations with their completely own versioning.
Author
There was a problem hiding this comment. i also think keeping functions in package sets is fine |
||
| - How do we handle aliases? | ||
quantenzitrone marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| - How do we handle versioned package sets? | ||
|
There was a problem hiding this comment. (Answering because this applies to idea 1) I generally think it's a good idea to think of Nixpkgs as having three types of package infrastructure systems:
Author
There was a problem hiding this comment.
Author
There was a problem hiding this comment. i want handle both 2 and 3 with this RFC |
||
| - How do we move large package sets? | ||
|
There was a problem hiding this comment. (Answering because this applies to idea 1) Some of them are easy enough. If we resolve the aforementioned issues with the python edge cases, it should be a simple matter of moving and renaming files. OCaml should also be pretty simple. The generated sets, like Haskell, Node, Lisp and R, will be impossible, though. Those packages don't actually exist as separate files in Nixpkgs, and are instead imported from some other upstream. This is something that again, we'll have to get the input of package set maintainers on. Moving everything over will end up being a case-by-case basis. |
||
|
|
||
| # Detailed design | ||
| [design]: #detailed-design | ||
|
|
||
| TODO: decide for one of the above ideas | ||
quantenzitrone marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| # Examples and Interactions | ||
| [examples-and-interactions]: #examples-and-interactions | ||
|
|
||
| TODO | ||
|
|
||
| # Drawbacks | ||
| [drawbacks]: #drawbacks | ||
|
|
||
| TODO | ||
|
|
||
| # Alternatives | ||
| [alternatives]: #alternatives | ||
|
|
||
| TODO: move other designs here once decided | ||
|
|
||
| # Prior art | ||
| [prior-art]: #prior-art | ||
|
|
||
| - https://github.com/NixOS/nixpkgs/issues/482537 | ||
| - https://github.com/NixOS/nixpkgs/issues/432625 | ||
| - `tclPackages` has their own `by-name` structure https://github.com/NixOS/nixpkgs/pull/344716 | ||
| - attempt to move `nushellPlugins` to `by-name` https://github.com/NixOS/nixpkgs/pull/482961 | ||
|
|
||
| # Unresolved questions | ||
| [unresolved]: #unresolved-questions | ||
|
|
||
| What parts of the design are still TBD or unknowns? | ||
|
|
||
| # Future work | ||
| [future]: #future-work | ||
|
|
||
| What future work, if any, would be implied or impacted by this feature without being directly part of the work? | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What is a package set? There are some that do not have a direct top-level name. I think in general some places of the text is written for a package set that comes with its set of expressions, while there are cases where meaningfully sets share a large fraction of the expressions.