`lein voom freshen` exits unsuccessfully with a status of 1

[dball]

~/src/pricing/pricing_ui[develop]$ lein voom freshen Fetching: /Users/dball/src/pricing/pricing_ui/.. (Li4=) Fetching: git@github.com:calvis/om-tools.git (Z2l0QGdpdGh1Yi5jb206Y2FsdmlzL29tLXRvb2xzLmdpdA==) Indexing /Users/dball/src/pricing/pricing_ui/.. commits 1/123 ...

It apparently fails to do the indexing on the pricing repo and exits with a status code of 1. Running the command again gives identical results. lein voom update-repo-dbs fails similarly.

Interestingly, this affects two developers, but not a third. We all claim to have the same lein voom version (the latest, "0.1.0-20160311_203101-g259fbfc"), the latest lein, the latest java.

I would appreciate any suggestions you have for debugging this.

[abrooks]

Thanks for filing this report.

First, can you email me (aaron@brooks1.net) the output (sanitized, if needed) of lein voom print-db <REPONAME> for each of the three developers noting which works and which doesn't? That helps give a lot of information.

Second, I have a few questions:

Is /Users/dball/src/pricing/pricing_ui/.. a bare repo (i.e. foo.git/) or one with a working copy (foo/.git/)? Using a non-bare repo as a remote is sketchy and can lead to problems. Am I reading the above snippet correctly in thinking that you have a local filesystem remote?

Have you moved the ~/.voom-repos/ aside and tried regenerating entirely new ~/.voom-repos/ structure? (Don't delete it, you should keep the original around so we can move it back for further debug.)

What OS versions / distros are the three devs using? It appears to be OS X but I don't want to assume that's true for all of them.

Thanks!

-A.

[dball]

The offending dependency seems to be:

             ^{:voom {:repo ".." :path "pricing_logic"
                      :sha "5a098dbb86c8d4eb6f9e2f0fc716b1889f33f73f"}}
             [sparkfund.internal/pricing-logic "0.1.0-20160701_143847-g5a098db"]

I'm not sure what the argument to print-db should be in the case, everything seems to fail with an exception:

~/src/pricing/pricing_ui[sfdc]$ lein voom print-db pricing-logic
java.io.FileNotFoundException: /Users/dball/.voom-repos/cHJpY2luZy1sb2dpYw==/.git/voomdb.frs.gz (No such file or directory)

If you let me know what arg I should use, I'll get that output to you.

The directory structure here is a little silly. We have two closely related projects we're developing that live in the same git repo: pricing_logic and pricing_ui. The pricing repo itself is a working copy.

I have blown away my ~/.voom-repos and started anew with fresh everythings, no change. We are all three on OSX.

[abrooks]

I strongly suspect the relative, local repo path as being the problem. The argument to 'print-db would be the repo as known by voom (usually an ssh or https URL). TBH, I have no idea if relative paths to local repos would work. (In fact, I'd expect it not to work.)

When two projects are in the same repo, just use the clone URL for freshening. This works fine. One of the problems voom was specifically trying to solve was multiple projects in the same repo so this is well-trod ground.

Does that help?

[dball]

You mentioned in email that the problem is likely to be the relative path to the repo. I changed that to a git url and continue experience the same issue, alas. I also don't get any output other than the file not found exception for any argument I've tried to lein voom print-db.

[abrooks]

Hm. lein voom print-db <URL> wants a URL that matches what voom knows. The easiest way to dump this would be with a command like this:

$ grep url ~/.voom-repos/*/.git/config

Then use the appropriate URL verbatim (excluding the "url =") with lein voom print-db. If you don't see the expected repo, this points to lein voom not knowing about your repo yet. You may need to be sure to do an extra lein voom build-deps in the depending project to be sure that you're able to fetch and build the existing dependency.

Can you show me the new metadata on the dependency?

Thanks!

[dball]

Sorry for the lapse, been heads down on other things.

Stubbornly, I keep getting:

~/src/api[add-pricing-queries]$ lein voom print-db git@github.com:SparkFund/pricing.git
java.io.FileNotFoundException: /Users/dball/.voom-repos/Z2l0QGdpdGh1Yi5jb206U3BhcmtGdW5kL3ByaWNpbmcuZ2l0/.git/voomdb.frs.gz (No such file or directory)
 at java.io.FileInputStream.open0 (FileInputStream.java:-2)
    java.io.FileInputStream.open (FileInputStream.java:195)
    java.io.FileInputStream.<init> (FileInputStream.java:138)
    clojure.java.io$fn__9520.invokeStatic (io.clj:229)
    clojure.java.io/fn (io.clj:229)
    clojure.java.io$fn__9433$G__9426__9440.invoke (io.clj:69)
    clojure.java.io$input_stream.invokeStatic (io.clj:136)
    clojure.java.io$input_stream.doInvoke (io.clj:121)
    clojure.lang.RestFn.invoke (RestFn.java:410)
    leiningen.voom$read_git_db.invokeStatic (voom.clj:1183)

[abrooks]

Can you provide the output of these commands?

• ls -1 ~/.voom-repos/
• grep url ~/.voom-repos/*/.git/config

Feel free to email this to me directly if you don't wish to show the names and locations of your repos.

[abrooks]

This is baffling. What you sent in the email is consistent. Thank you for your continued patience as we figure this out.

What is the output of these commands (in order)? (You can send publicly or privately as you please)

• lein voom fetch-all
• lein voom update-repo-dbs
• ls -ls /Users/dball/.voom-repos/Z2l0QGdpdGh1Yi5jb206U3BhcmtGdW5kL3ByaWNpbmcuZ2l0/.git/
• lein voom print-db git@github.com:SparkFund/pricing.git # Again, I know. :)

[dball]

One possibly useful observation: I'm on OSX 10.11.6, while the developer for whom lein voom works in this configuration is on OSX 10.10.4.

[abrooks]

Interesting. What does the output of git --version say across all of the developers machines (noting the one that works and the ones that don't).

[dball]

I've explored this a bit. I've been on git 2.9.x installed by homebrew, I'm currently on git version 2.7.4 (Apple Git-66). The developer for whom everything works is on git 2.3.1, which is immediately suggestive... but I installed git 2.3.1 from source, blew away my .voom-repos and tried again with it, to no apparent avail.

[abrooks]

Continuing digging deeper, can you create a file ~/tmp/bin/git with this content (and chmod u+x ~/tmp/bin/git):

#!/bin/bash

LOG_FILE=~/lein-git-events--${PPID}.log

echo "PWD: $PWD" >> ${LOG_FILE}
echo "git" "$@" >> ${LOG_FILE}
PATH="${PATH#*:}" "git" "$@"
echo ">> Exited with status: $?" >> ${LOG_FILE}

and then run it with this command:

PATH=~/tmp/bin:$PATH lein voom update-repo-dbs; echo "lein exit status:" $?

Thanks!

and then email me the output ~/lein-git-events--<PID>.log file and lein exit status?

[abrooks]

[trace ouptut received via email]

Everything (aside from the noted ".."/"Li4=/" repo path that needs to be expunges) looks fine on the git side.

We're pulling out the big hammer — full tracing of the leiningen.voom namespace. Please save the below tracing code from @Chouser and I in a file ~/tmp/lein-voom-trace/user.clj:

(prn :loading-patch)
(require '[cemerick.pomegranate :as p])
(p/add-dependencies :coordinates '[[org.clojure/tools.trace "0.7.9"]])
(require '[clojure.tools.trace :as trace])

(alter-var-root
 #'trace/tracer
 (constantly
  (fn [name value]
    (print (str "TRACE " (Thread/currentThread) " " (when name (str " " name)) ": " value "\n"))
    (flush))))

(alter-var-root
 #'clojure.core/require
 (fn [orig-require]
   (fn new-require [& args]
     (doto (apply orig-require args)
       (do
         (when (some #(= 'leiningen.voom
                         (if (coll? %) (first %) %)) args)
           (trace/trace-ns* 'leiningen.voom)
           (prn :applied-trace)))))))
(prn :loaded-patch)

To enable this tracing run in the following manner:

CLASSPATH=~/tmp/lein-voom-trace/ lein voom update-repo-dbs | tee ~/lein-voom-trace-events.log

If you're concerned about leaked project details, scan the file and replace any sensitive strings with unique replacements (i.e. sed 's/SENSITIVE1/redacted001/g', sed 's/SENSITIVE2/redacted002/g') so we can still see the structure. Once you've done any redacting (if needed) send the log file to me via email.

Thanks!

[abrooks]

@dball Just checking in to see if this issue is still persisting or if you had a chance to run the more detailed tracing. Hope all is well!