Nixpkgs fetchers

[WeetHet]

Currently, npins only supports builtin fetchers, as it's mostly focused on nix dependency pinning. I was wondering, however, since it is an outside project, it can have a half-dependency on nixpkgs, without creating a loop or breaking abstraction. Therefore, I propose adding a flag to enable fetching a source using nixpkgs fetchers (which would rely on nixpkgs being in the sources.json). This would allow for external build-time dependency management and be extremely beneficial as it would allow replacing hand-written update scripts I (and most likely a lot of other people) have for this purpose

[andir]

That is similar to an idea that I implemented in #40

[piegamesde]

As a workaround, you can easily write your own version of the default.nix which relies on the Nixpkgs fetchers. Heck, maybe you could even use scopedImport to override the builtins fetchers to point to the Nixpkgs fetchers instead

[piegamesde]

I'm increasingly interested in this as well, especially because of lockfile mode. Thinking of it, if you already have a pkgs then simply having the pin reading code attached to pkgs.npins sounds like the best option to me. Though it won't be the easiest on the maintenance side, need to figure out that still

[piegamesde]

let pkgs = import <nixpkgs> {};
in
builtins.scopedImport {
  builtins = builtins // {
    fetchTarball = { url, sha256 }: pkgs.fetchzip { inherit url sha256; };
    fetchurl = { url, sha256 }: pkgs.fetchurl { inherit url sha256; };
    fetchGit = { url, submodules, rev, name, sha256}: pkgs.fetchgit { inherit url rev name sha256; fetchSubmodules = submodules; };
  };
} ./npins/default.nix

This is the direction I'm currently experimenting with. Yes this is a cursed hacky hack, but on the other side it seems to be the best option, maintenance wise.

The main issue is that I cannot easily make default.nix more generic due to backwards compatibility constraints, but also I do not want to duplicate any of its logic.

[winterqt]

I'm increasingly interested in this as well, especially because of lockfile mode.

What is the benefit of this in lockfile mode if the actual default.nix isn’t used in that case? I imagine you mean just because the hashes may need to be calculated differently due to possible differences between builtins and Nixpkgs’ fetchers?

[piegamesde]

@winterqt I am building a data pipeline with source repositories as the input. Having the inputs as proper derivations allows me to easily do remote builds. Otherwise I'd have to fetch everything on the main machine and that would slow things down considerably. The hashes should absolutely be the same, and things will horribly blow up should they ever change (it already happened, was not fun to debug).

Also, lockfile mode is not about not using the default.nix, it is about not being tied to a specific directory layout. I tend to have a normal npins setup in a repository and then several additional lockfiles with the actual data for the pipeline