Index corrupted after `git status` with Git 2.40.0 and 2.40.1

[stanio]

• I was not able to find an open or closed issue matching what I'm seeing

Setup

• Which version of Git for Windows are you using? Is it 32-bit or 64-bit?

$ git --version --build-options

git version 2.40.1.windows.1
cpu: x86_64
built from commit: ceee26d5cac05a3437097b43d034c4ad2e99d571
sizeof-long: 4
sizeof-size_t: 8
shell-path: /bin/sh
feature: fsmonitor--daemon

• Which version of Windows are you running? Vista, 7, 8, 10? Is it 32-bit or 64-bit?

$ cmd.exe /c ver

Microsoft Windows [Version 10.0.19045.2846]

• What options did you set as part of the installation? Or did you choose the defaults?

# One of the following:
> type "C:\Program Files\Git\etc\install-options.txt"
> type "C:\Program Files (x86)\Git\etc\install-options.txt"
> type "%USERPROFILE%\AppData\Local\Programs\Git\etc\install-options.txt"
> type "$env:USERPROFILE\AppData\Local\Programs\Git\etc\install-options.txt"
$ cat /etc/install-options.txt

Editor Option: VIM
Custom Editor Path:
Default Branch Option:
Path Option: Cmd
SSH Option: OpenSSH
Tortoise Option: false
CURL Option: OpenSSL
CRLF Option: LFOnly
Bash Terminal Option: MinTTY
Git Pull Behavior Option: Merge
Use Credential Manager: Enabled
Performance Tweaks FSCache: Enabled
Enable Symlinks: Enabled
Enable Pseudo Console Support: Enabled
Enable FSMonitor: Disabled

• Any other interesting things about your environment that might be related to the issue you're seeing?

I don't know what could be the cause but I'm observing this problem only on a particular laptop/Windows setup. I can't reproduce it on two other laptops (Windows 10 and Windows 11). I'm not observing the problem on the former laptop using Git for Windows 2.39.2, also – git reset . using that version appears to correct the previously (2.40.x) corrupted index (so I've reverted to it, for the time being). I'm consistently observing the problem after I upgrade to Git for Windows 2.40.x on that laptop.

Details

• Which terminal/shell are you running Git from? e.g Bash/CMD/PowerShell/other

I'm mainly using Git Bash in Windows Terminal, but I've tried it happens using CMD in Windows Terminal and in the legacy Console, and Git Bash in MinTTY.

• What commands did you run to trigger this issue? If you can provide a Minimal, Complete, and Verifiable example this will help us understand the issue.

~/work
$ git clone https://github.com/stanio/xbrz-java.git xbrz-java.test

~/work
$ cd xbrz-java.test

~/work/xbrz-java.test master ≡
$ git status
On branch master
Your branch is up to date with 'origin/master'.

nothing to commit, working tree clean

# The corruption has already happened at this stage
~/work/xbrz-java.test master ≡  -6
$ git status
On branch master
Your branch is up to date with 'origin/master'.

Changes to be committed:
  (use "git restore --staged <file>..." to unstage)
        deleted:    .gitignore
        deleted:    CODE_OF_CONDUCT.md
        deleted:    LICENSE.md
        deleted:    PUBLISHING.md
        deleted:    README.md
        deleted:    pom.xml

~/work/xbrz-java.test master ≡  -6
$ git status
On branch master
Your branch is up to date with 'origin/master'.

Changes to be committed:
  (use "git restore --staged <file>..." to unstage)
        deleted:    .gitignore
        deleted:    CODE_OF_CONDUCT.md
        deleted:    LICENSE.md
        deleted:    PUBLISHING.md
        deleted:    README.md
        deleted:    pom.xml

~/work/xbrz-java.test master ≡  -6
$ git reset .

~/work/xbrz-java.test master ≡  -6
$ git status
On branch master
Your branch is up to date with 'origin/master'.

Changes to be committed:
  (use "git restore --staged <file>..." to unstage)
        deleted:    .gitignore
        deleted:    CODE_OF_CONDUCT.md
        deleted:    LICENSE.md
        deleted:    PUBLISHING.md
        deleted:    README.md
        deleted:    pom.xml

Untracked files:
  (use "git add <file>..." to include in what will be committed)
        .gitignore
        CODE_OF_CONDUCT.md
        LICENSE.md
        PUBLISHING.md
        README.md
        docs/
        pom.xml
        src/
        xbrz-awt/
        xbrz-core/
        xbrz-tool/

• What did you expect to occur after running these commands?

$ git status
On branch master
Your branch is up to date with 'origin/master'.

nothing to commit, working tree clean

• What actually happened instead?

(see the commands and their output, which I've given as "steps to reproduce")

• If the problem was occurring with a specific repository, can you provide the URL to that repository to help us with testing?

Happens with all repositories I have checked out on that laptop. Here's .git snapshots from stanio/xbrz-java:

• 0.git.zip – fresh clone
• 1.git.zip – after git status
• 2.git.zip – after git reset .

The original .git/index appears to get smaller (missing info?) and a .git/sharedindex.0000000000000000000000000000000000000000 appears.

[derrickstolee]

Hi, @stanio. Sorry you're having trouble here.

~/work
$ git clone https://github.com/stanio/xbrz-java.git xbrz-java.test

~/work
$ cd xbrz-java.test

~/work/xbrz-java.test master ≡
$ git status
On branch master
Your branch is up to date with 'origin/master'.

nothing to commit, working tree clean

# The corruption has already happened at this stage
~/work/xbrz-java.test master ≡  -6
$ git status
On branch master
Your branch is up to date with 'origin/master'.

Changes to be committed:
  (use "git restore --staged <file>..." to unstage)
        deleted:    .gitignore
        deleted:    CODE_OF_CONDUCT.md
        deleted:    LICENSE.md
        deleted:    PUBLISHING.md
        deleted:    README.md
        deleted:    pom.xml

I'm not able to reproduce this, but I'm using a slightly modified version of v2.40.1.windows.1. I don't think it should matter, but worth mentioning.

The original .git/index appears to get smaller (missing info?) and a .git/sharedindex.0000000000000000000000000000000000000000 appears.

This shouldn't happen unless you have core.splitIndex enabled. Is that the case on your system?

Even with that, the fact that the trailing hash is all zeroes is a problem. So the bug lies in the split index somewhere, but it's strange how it would manifest like this.

Have you been able to try other Git for Windows versions and have this reproduce? Any chance we can bisect at least to a Git version? (or are you sure this doesn't reproduce with v2.39.3?)

[stanio]

Hi, @derrickstolee. Thank you very much for your fast and helpful response:

This shouldn't happen unless you have core.splitIndex enabled. Is that the case on your system?

Yes, that appears to be the case:

$ git config --list --show-scope
...
global  core.splitindex=true
...

I don't really remember when and for what purpose I've set it. Once I've removed it from my user config the problem has gone away.

Have you been able to try other Git for Windows versions and have this reproduce? Any chance we can bisect at least to a Git version? (or are you sure this doesn't reproduce with v2.39.3?)

I'm sure it is not happening with v2.39.2 – I've tried at least a dozen times before submitting this report. I first noticed the problem with the v2.40.0 upgrade (some time ago). The outcome got me panicked and I reverted to v2.39.2. git reset . then appears to have corrected the corrupted index. A few days ago I got an automated v2.40.1 install and I've started observing the same corruption.

[stanio]

This shouldn't happen unless you have core.splitIndex enabled. Is that the case on your system?

Once I've removed it from my user config the problem has gone away.

FWIW, I've set it to true on another laptop I wasn't observing the problem on, and I'm still not seeing the problem there. In the .git directory I'm seeing files:

index
sharedindex.f335c9d0488aad722f6dd0bbf48d63e4d8d58b63

[stanio]

This shouldn't happen unless you have core.splitIndex enabled. Is that the case on your system?

Once I've removed it from my user config the problem has gone away.

Then, on the problem laptop, the problem appears solved only for new clones. git reset . or git reset --hard doesn't appear to clear the corrupted index (completely). With git reset --hard I'm also observing:

$ git reset --hard

$ git status
On branch master
Your branch is up to date with 'origin/master'.

Changes to be committed:
  (use "git restore --staged <file>..." to unstage)
        deleted:    gradlew
        deleted:    gradlew.bat

Untracked files:
  (use "git add <file>..." to include in what will be committed)
        gradle/
        src/

$ git reset --hard
fatal: Unable to create 'C:/Users/stanio/work/diff-view.test/.git/index.lock': File exists.

Another git process seems to be running in this repository, e.g.
an editor opened by 'git commit'. Please make sure all processes
are terminated then try again. If it still fails, a git process
may have crashed in this repository earlier:
remove the file manually to continue.

$ rm .git/index.lock

Performing git reset --hard once again – first completes seemingly successfully, then shows the error for the existing index.lock. No processes appear to hold handles to that file (verified with File Locksmith power-toy as well).

[stanio]

git reset . or git reset --hard doesn't appear to clear the corrupted index (completely).

git restore --staged . did it for me, at the end.

FWIW:

$ git config --global core.splitIndex true

$ git status

reproduces the corruption on that laptop, again... and reinstalling Git 2.39.2 corrects the problem.

[stanio]

I've looked at my other settings in ~/.gitconfig and I've found these two:

[feature]
	manyFiles = true
	experimental = true

in combination with core.splitIndex = true to be causing the problem. Don't remember when and for what purpose I had them in there.

[dscho]

I believe that this will fix it: gitgitgadget#1497. It has not made it into any official Git version yet, and it is expected to be shipped with v2.41 (see https://gh.io/gitCal for the timeline).

[derrickstolee]

@dscho: thanks for pointing out those bugs and the fix. Do you have a Git for Windows snapshot that includes that fix? I noticed that the nightly snapshots are not built from shears/main, for example.

[rimrul]

Do you have a Git for Windows snapshot that includes that fix? I noticed that the nightly snapshots are not built from shears/main, for example.

We tend to only rebase onto upstream Git after (pre-)releases. The changes have not made it into any Git for Windows snapshots. We'd need to cherry-pick them if we want to fast-track them into the Git for Windows snapshots before 2.41.0-rc0.

[dscho]

I agree that it "would be nice if..." we had some automation to quickly generate artifacts for testing such patches.

The truth is that we will have this sort of automation. We already moved the regular Git for Windows release process from Azure Pipelines to GitHub workflows, and part of the automation that performs the snapshot builds will be moved from Azure Pipelines to GitHub workflows soon, too. At that point it will be a doozy to address aforementioned wish on top of this automation.

[stanio]

Trying with Git 2.41.0.windows.1 I'm not observing the corruption while having all of:

[core]
	splitindex = true
[feature]
	manyfiles = true
	experimental = true

set.

[dscho]

Yay! Closing time!