Running brew endlessly spawns git processes until killed

[mike-hearn]

brew config output

brew does not run, so unfortunately I cannot run brew config

brew doctor output

See above.

• The brew doctor above contains no "Warning" lines.

What were you trying to do (and why)?

I was trying to run brew.

I have reproduced this problem on both my M1 Macbook Air and now my Intel iMac. On the iMac, brew was working fine until I ran brew update and now brew hangs once again. The symptoms are identical on both machines.

What happened (include all command output)?

See screen recording below.

On the right pane, I'm running brew. On the left, an htop window filtered to all processes containing brew.

When I run brew, it spawns new git processes endlessly until killed.

What did you expect to happen?

I expected brew to work w/o issue.

Step-by-step reproduction instructions (by running brew commands)

Other than updating brew to the latest version, the steps to reproduce are:

1. Run brew

My sense is this is something to do with my machine configuration, but it's confusing that it happened with two separate machines, and especially concerning that it was fine on my iMac until updating to the latest Homebrew.

[carlocab]

Can you try brew update-reset, in case that works?

If it doesn't, what is the output of

which -a git
git --version

?

[mike-hearn]

brew update-reset causes the same git process infinite loop, so no output.

❯ which -a git
/usr/local/bin/git
/usr/bin/git

~
❯ git --version
git version 2.28.0

I will also add that if I go to /usr/local/Homebrew and run git checkout 3.0.2, brew works again. On 3.0.3 it does not:

~
❯
Homebrew on  stable
❯ git checkout 3.0.2
Note: switching to '3.0.2'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at b712bcc01 Merge pull request #10670 from Bo98/nounzip-stage-fix

Homebrew on  HEAD (b712bcc)
❯ brew
Example usage:
  brew search [TEXT|/REGEX/]
  brew info [FORMULA...]
  brew install FORMULA...
  brew update
  brew upgrade [FORMULA...]
  brew uninstall FORMULA...
  brew list [FORMULA...]

Troubleshooting:
  brew config
  brew doctor
  brew install --verbose --debug FORMULA

Contributing:
  brew create [URL [--no-fetch]]
  brew edit [FORMULA...]

Further help:
  brew commands
  brew help [COMMAND]
  man brew
  https://docs.brew.sh

Homebrew on  HEAD (b712bcc)
❯ git checkout 3.0.3
Previous HEAD position was b712bcc01 Merge pull request #10670 from Bo98/nounzip-stage-fix
HEAD is now at 9d2ee344f Merge pull request #10749 from Homebrew/dependabot/bundler/Library/Homebrew/minitest-5.14.4

Homebrew on  HEAD (9d2ee34)
❯ brew

[...hangs until killed]

[carlocab]

I will also add that if I go to /usr/local/Homebrew and run git checkout 3.0.2, brew works again.

That's troubling. Can you do a git bisect to try to pin down a specific commit?

This is also weird:

❯ which -a git
/usr/local/bin/git
/usr/bin/git

~
❯ git --version
git version 2.24.3 (Apple Git-128)

On my system:

❯ which -a git && git --version
/usr/local/bin/git
/usr/bin/git
git version 2.30.1

Your calls to git seem to be picking up the built-in one, and not the brewed one:

❯ /usr/bin/git --version
git version 2.24.3 (Apple Git-128)

I don't see how it's related to the issue you're having, if at all, though.

[jonchang]

We've been trying to debug this internally for the last hour or so as it's affecting certain Github Actions we're using. This is the problem commit:

% git bisect bad
7e75a49b772f03f1373c6b3184a354a292693728 is the first bad commit
commit 7e75a49b772f03f1373c6b3184a354a292693728
Author: Bo Anderson <mail@boanderson.me>
Date:   Thu Feb 11 16:30:14 2021 +0000
    shims/mac/super: add ruby shims to set SDKROOT

[mike-hearn]

Your calls to git seem to be picking up the built-in one, and not the brewed one:

I actually edited that output before you commented. That was a mistake on my end. It was only picking up the Apple version of git because I had messed with my PATH moments before running it while I was doing some debugging on my own. Sorry for the confusion there.

Annnnnyway, I had never used git bisect before and wow, it's awesome. Looks like this is the commit that triggered this bug on my machines:

❯ git bisect bad
7e75a49b772f03f1373c6b3184a354a292693728 is the first bad commit
commit 7e75a49b772f03f1373c6b3184a354a292693728
Author: Bo Anderson <mail@boanderson.me>
Date:   Thu Feb 11 16:30:14 2021 +0000

    shims/mac/super: add ruby shims to set SDKROOT

 .github/workflows/tests.yml           |   2 +-
 Library/Homebrew/shims/mac/super/gem  |   1 +
 Library/Homebrew/shims/mac/super/rake |   1 +
 Library/Homebrew/shims/mac/super/ruby |  12 ++++
 Library/Homebrew/shims/scm/git        | 107 ++++------------------------------
 Library/Homebrew/shims/utils.sh       |  94 +++++++++++++++++++++++++++++
 6 files changed, 121 insertions(+), 96 deletions(-)
 create mode 120000 Library/Homebrew/shims/mac/super/gem
 create mode 120000 Library/Homebrew/shims/mac/super/rake
 create mode 100755 Library/Homebrew/shims/mac/super/ruby

[jonchang]

#10763

[mike-hearn]

I'm not a bash expert, but if I remove this block from https://github.com/Homebrew/brew/blob/master/Library/Homebrew/shims/scm/git, it runs. Is this causing an infinite loop somehow?

brew/Library/Homebrew/shims/scm/git

Lines 6 to 10 in f43cc96

	if [ -z "$HOMEBREW_LIBRARY" ]
	then
	echo "${0##*/}: This shim is internal and must be run via `brew`." >&2
	exit 1
	fi

[Bo98]

Yeah those backticks will cause the problem. It would be interesting to know why $HOMEBREW_LIBRARY is unset though - it should always be set at that point.

[mike-hearn]

My PATH prefers the Homebrew bash which is 5.0.18(1)-release (x86_64-apple-darwin19.5.0), but that script specifies /bin/bash/ which is 3.2.57(1)-release (x86_64-apple-darwin20) that comes with MacOS.

[Bo98]

The issue was HOMEBREW_NO_ENV_FILTERING, which we don't really document or support. Our CI doesn't test it so it went unnoticed.

[jacktose]

FYI I had this problem with linuxbrew (Homebrew/discussions#949), and I use HOMEBREW_NO_ENV_FILTERING (because of an old issue: Linuxbrew/brew#736 (comment)). This is now fixed, presumably by #10763.

[Bo98]

My general recommendation is that if anyone is currently using HOMEBREW_NO_ENV_FILTERING, then please open an issue detailing what's broken without it and we can see what else we can do to help your situation.

At this point, HOMEBREW_NO_ENV_FILTERING should really not be used beyond temporary troubleshooting. We've considered gating it behind HOMEBREW_DEVELOPER or removing it entirely.