Slow performance of intersection queries

[spadgos]

I work on a rather large monorepo (~550k commits, 45gb on disk) and have noticed an issue with trying to filter my branches by date. I will often end up with some experimental branches which I'm no longer keeping up to date, but do want to hang onto for future reference. Currently these clog up my smartlog output, so I thought I'd filter them by date:

$ git sl 'branches() and author.date("after:1 month ago")'

This gives the output I'd like, but has the unfortunate side-effect of being very slow. Here's some timings I ran:

• git-branchless wrap -- query 'branches()' > /dev/null 0.05s user 0.02s system 89% cpu 0.087 total
• git-branchless wrap -- query 'author.date("after:1 month ago")' > /dev/null 19.26s user 4.82s system 435% cpu 5.526 total
• git-branchless wrap -- query 'branches() and author.date("after:1 month ago")' 18.20s user 4.53s system 510% cpu 4.457 total

I imagine behind the scenes, the intersection (and) gathers both revsets and then performs the intersection. I wonder if there's an optimization here where the second filter is only applied to the first revset (if that makes sense).

Alternatively, is there a way that the branches() selector could also support a date filter?

[claytonrcarter]

the intersection (and) gathers both revsets and then performs the intersection

This is correct, more or less. I've also been bothered by this, even though I work in repos multiple orders of magnitudes smaller that yours, and I had patched it locally but hadn't pushed it until just now. Are you able to build from source? If so, can you try out #1591 to see if helps you?

The main caveat with that change is that it is fairly simplistic: it assumes that the lhs is the smaller set, so (eg) 'branches() and author.date("after:1 month ago")' would be fast (because it would only check the branch commits that are newer than a month) but 'author.date("after:1 month ago") and branches()' would probably not be much faster, because it would still check all commits to find those more recent than 1 month, and then extract the branch commits from that.

[spadgos]

wrt the order of the sets being important, I think this might be understandable to most programmers as it's similar to if (cheapOperation() && expensiveOperation()). It's been a looong time since I wrote SQL in anger, but I seem to recall that MySQL also has different performance behavior depending on the order of conditions for the same reason.

fwiw, I found a terrible workaround: run a query to find the old branches and write that to my git config as an alias.

$ git config set --global \
  branchless.revsets.alias.old-branches \
  "`git query 'branches() and author.date("before:1 month ago")' -b | paste -sd "+" -`"

This creates a line in my gitconfig like:

[branchless "revsets.alias"]
  old-branches=an-old-branch+another-old-branch+and-so-on

and then I can use that in other aliases:

$ git sync --pull 'draft() - old-branches()'