error prefetching hash: exit status 1

[lilyball]

I'm trying to use this on a project with a Gopkg.lock and when I run dep2nix I get the output

Found 22 projects to process.
* Processing: "[redacted]/common/golibs"
error prefetching hash: exit status 1

The project is an internal project but the import domain doesn't require authentication AFAICT (it just requires being on VPN, which I am). I don't know what "error prefetching hash" means. The Gopkg.lock file lists this for the given dependency:

[[projects]]
  digest = "1:8b59f86796b3cf384bde6c7b3ab2da9fa4c7eff057d216ccd420c80894a9c36f"
  name = "[redacted]/common/golibs"
  packages = [
    "bininfo",
    "pkgpath",
  ]
  pruneopts = "NUT"
  revision = "c3c7c6b379004afc72b5a25e220515c2da72d9e4"

[lilyball]

If it makes a difference, dep ensure works just fine.

[lilyball]

I'm still running into this. I really wish I could get this tool to at least print out the command it was executing.

[lilyball]

I just modified the source to print out the nix-prefetch-git args and the local path it's trying to prefetch from doesn't exist.

Edit: Actually it does exist while the tool is running, I guess it deletes the dir before returning.

[lilyball]

I figured out that stderr was being lost. After restoring that, I got the wonderful error

error: The path name 'git-ssh---git@redacted.example.com-common-golibs-c3c7c6b' is invalid: the '@' character is invalid. Path names are alphanumeric and can include the symbols +-._?= and must not begin with a period. Note: If 'git-ssh---git@redacted.example.com-common-golibs-c3c7c6b' is a source file and you cannot rename it on disk, builtins.path { name = ... } can be used to give it an alternative name.

[lilyball]

Ultimately, the problem is dep produces a git+ssh://git@redacted.example.com/common/golibs URL, which it checks out on disk as /tmp/935322867/sources/git-ssh---git@redacted.example.com-common-golibs-c3c7c6b, and nix-prefetch-git then takes the basename of that and tries to use that as the name in the nix store.

[lilyball]

This is rather hacky but seems to work:

diff --git a/prefetch.go b/prefetch.go
index dc3e74c..78b3b00 100644
--- a/prefetch.go
+++ b/prefetch.go
@@ -4,6 +4,9 @@ import (
 	"bytes"
 	"encoding/json"
 	"github.com/Masterminds/vcs"
+	"io/ioutil"
+	"net/url"
+	"os"
 	"os/exec"
 	"strings"
 )
@@ -36,8 +40,31 @@ func cmdStdout(command string, arguments ...string) (string, error) {
 
 type gitPrefetcher struct{}
 
-func (p *gitPrefetcher) fetchHash(url string, revision string) (string, error) {
-	out, err := cmdStdout("nix-prefetch-git", "--url", url, "--rev", revision, "--quiet", "--fetch-submodules")
+func (p *gitPrefetcher) fetchHash(localUrl string, revision string) (string, error) {
+	if strings.Contains(localUrl, "@") {
+		// nix-prefetch-git does not sanitize the @ character, but it's not allowed in Nix paths.
+		// We need to copy this directory to another path that doesn't contain the @.
+		realUrl, err := url.Parse(localUrl)
+		if err != nil {
+			return "", err
+		}
+		dir, err := ioutil.TempDir("", "dep2nix")
+		if err != nil {
+			return "", err
+		}
+		defer os.RemoveAll(dir) // clean up the temp dir
+		// HACK: I don't want to write a recursive copy myself. I'll just leverage ditto instead.
+		cmd := exec.Command("ditto", "--", realUrl.EscapedPath(), dir)
+		cmd.Stderr = os.Stderr
+		if err := cmd.Run(); err != nil {
+			return "", err
+		}
+		localUrl = (&url.URL{
+			Scheme: "file",
+			Path:   dir,
+		}).String()
+	}
+	out, err := cmdStdout("nix-prefetch-git", "--url", localUrl, "--rev", revision, "--quiet", "--fetch-submodules")
 	if err != nil {
 		return "", err
 	}

[lilyball]

It looks like we could use the --out flag to nix-prefetch-git instead, though I haven't tested that yet.