Git v2.40.0 crashing with "could not read index: invalid data in index - calculated checksum does not match expected"

[mlanln]

Description of the bug

I am trying out git-branchless in a repo at work. git sl works fine but write commands like git record, git amend, and git reword crash.

Expected behavior

No error

Actual behavior

Error

$ git record -m test
The application panicked (crashed).
Message:  A fatal error occurred: 
   0: could not read index: invalid data in index - calculated checksum does not match expected; class=Index (10)
   1: invalid data in index - calculated checksum does not match expected; class=Index (10)

Location:
   /home/mlanln/.cargo/registry/src/github.com-1ecc6299db9ec823/git-branchless-record-0.7.0/src/lib.rs:92

Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets.
Location: /home/mlanln/.cargo/registry/src/github.com-1ecc6299db9ec823/git-branchless-0.7.0/src/commands/mod.rs:217

Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets.

Version of rustc

rustc 1.65.0

Automated bug report

Software version

git-branchless 0.7.0

Operating system

Linux 5.15.0-1031-gcp

Command-line

/home/mlanln/.cargo/bin/git-branchless bug-report 

Environment variables

SHELL=/bin/bash
EDITOR='code -w'

Git version

> git version 
git version 2.40.0

Hooks

Show 7 hooks

Hook post-applypatch

#!/bin/sh
## START BRANCHLESS CONFIG

git branchless hook post-applypatch "$@"

## END BRANCHLESS CONFIG

Hook post-checkout

#!/bin/sh
## START BRANCHLESS CONFIG

git branchless hook post-checkout "$@"

## END BRANCHLESS CONFIG

Hook post-commit

#!/bin/sh
## START BRANCHLESS CONFIG

git branchless hook post-commit "$@"

## END BRANCHLESS CONFIG

Hook post-merge

#!/bin/sh
## START BRANCHLESS CONFIG

git branchless hook post-merge "$@"

## END BRANCHLESS CONFIG

Hook post-rewrite

#!/bin/sh
## START BRANCHLESS CONFIG

git branchless hook post-rewrite "$@"

## END BRANCHLESS CONFIG

Hook pre-auto-gc

#!/bin/sh
## START BRANCHLESS CONFIG

git branchless hook pre-auto-gc "$@"

## END BRANCHLESS CONFIG

Hook reference-transaction

#!/bin/sh
## START BRANCHLESS CONFIG

# Avoid canceling the reference transaction in the case that `branchless` fails
# for whatever reason.
git branchless hook reference-transaction "$@" || (
echo 'branchless: Failed to process reference transaction!'
echo 'branchless: Some events (e.g. branch updates) may have been lost.'
echo 'branchless: This is a bug. Please report it.'
)

## END BRANCHLESS CONFIG

Events

Show 5 events

Event ID: 24, transaction ID: 21 (message: hook-post-checkout)

1. RefUpdateEvent { timestamp: 1681181943.1674395, event_tx_id: EventTransactionId(21), ref_name: ReferenceName("HEAD"), old_oid: 3900bee7374c2fa4318d0b30056db8ac147c0408, new_oid: 3900bee7374c2fa4318d0b30056db8ac147c0408, message: None }

:
O b227894 14d (main, redacted-ref-0) xxx xxxxxxx xx xxxxxxxxx xxxxxxx xxxxxxxxxx xxxxxx
|
@ 3900bee 4h (redacted-ref-1, redacted-ref-2) xxxxxxx xxxxxx xxx xxxxxxxxxx xxxxx xxxxxxxxxx

Event ID: 23, transaction ID: 20 (message: reference-transaction)

1. RefUpdateEvent { timestamp: 1681181943.1404214, event_tx_id: EventTransactionId(20), ref_name: ReferenceName("refs/heads/redacted-ref-1"), old_oid: 0000000000000000000000000000000000000000, new_oid: 3900bee7374c2fa4318d0b30056db8ac147c0408, message: None }

:
O b227894 14d (main, redacted-ref-0) xxx xxxxxxx xx xxxxxxxxx xxxxxxx xxxxxxxxxx xxxxxx
|
@ 3900bee 4h (redacted-ref-1, redacted-ref-2) xxxxxxx xxxxxx xxx xxxxxxxxxx xxxxx xxxxxxxxxx

Event ID: 22, transaction ID: 19 (message: hook-post-rewrite)

1. RewriteEvent { timestamp: 1681164709.6778862, event_tx_id: EventTransactionId(19), old_commit_oid: 89b215533a6ef3787065f6471f7b6ceacd6de0e8, new_commit_oid: 3900bee7374c2fa4318d0b30056db8ac147c0408 }

:
O b227894 14d (main, redacted-ref-0) xxx xxxxxxx xx xxxxxxxxx xxxxxxx xxxxxxxxxx xxxxxx
|
@ 3900bee 4h (redacted-ref-1, redacted-ref-2) xxxxxxx xxxxxx xxx xxxxxxxxxx xxxxx xxxxxxxxxx

Event ID: 21, transaction ID: 18 (message: post-commit)

1. CommitEvent { timestamp: 1681164705.0, event_tx_id: EventTransactionId(18), commit_oid: NonZeroOid(3900bee7374c2fa4318d0b30056db8ac147c0408) }

:
O b227894 14d (main, redacted-ref-0) xxx xxxxxxx xx xxxxxxxxx xxxxxxx xxxxxxxxxx xxxxxx
|
@ 3900bee 4h (redacted-ref-1, redacted-ref-2) xxxxxxx xxxxxx xxx xxxxxxxxxx xxxxx xxxxxxxxxx

Event ID: 19, transaction ID: 17 (message: reference-transaction)

1. RefUpdateEvent { timestamp: 1681164709.6239216, event_tx_id: EventTransactionId(17), ref_name: ReferenceName("HEAD"), old_oid: 89b215533a6ef3787065f6471f7b6ceacd6de0e8, new_oid: 3900bee7374c2fa4318d0b30056db8ac147c0408, message: None }
2. RefUpdateEvent { timestamp: 1681164709.6239216, event_tx_id: EventTransactionId(17), ref_name: ReferenceName("refs/heads/redacted-ref-2"), old_oid: 89b215533a6ef3787065f6471f7b6ceacd6de0e8, new_oid: 3900bee7374c2fa4318d0b30056db8ac147c0408, message: None }

:
O b227894 14d (main, redacted-ref-0) xxx xxxxxxx xx xxxxxxxxx xxxxxxx xxxxxxxxxx xxxxxx
|
@ 3900bee 4h (redacted-ref-1, redacted-ref-2) xxxxxxx xxxxxx xxx xxxxxxxxxx xxxxx xxxxxxxxxx

Version of git-branchless

No response

Version of git

No response

[arxanas]

Hi @mlanln, sorry to hear that this is happening. Another user also reported this on Discord, but unfortunately we were unable to find a solution; they ended up recloning the repo. My assumption is that this is a bug in libgit2, but I'm not sure.

[mlanln]

Darn, I was hoping it was fixable. I get the same error on a newly git inited repo so recloning doesn't change anything.

[arxanas]

@mlanln can you try with a version of Git earlier than 2.40? Maybe something about the index format changed and libgit2 hasn't upgraded to handle it yet.

[mlanln]

Bingo! I downgraded to git version 2.39.2 and all of those commands are working now. Thanks!!

[arxanas]

@mlanln Thanks for confirming. I'll leave the issue open in case others run into the same problem. I also very recently updated our version of git2 (in f25babf); if you have time, could you try with that version of the code and see if the problem still occurs with Git v2.40? (See Manual testing.)

[auscompgeek]

I'm still seeing this error with git v2.40.0 with f25babf. Here's me trying to git move -b my-branch:

Attempting rebase in-memory...
The application panicked (crashed).
Message:  A fatal error occurred:
   0: could not cherry-pick commit 7e930e0adb6ab058e10d7ddad87a5733ada76400 onto 273ed0e27fa37375a066e047916432f76e430ae2: invalid data in index - calculated checksum does not match expected; class=Index (10)
   1: invalid data in index - calculated checksum does not match expected; class=Index (10)

Location:
   /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/core/src/convert/mod.rs:726

Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets.
Location: git-branchless/src/commands/mod.rs:221

[arxanas]

It looks like it's this issue in libgit2: libgit2/libgit2#6531

You can follow the workaround here:

workaround is to disable the offending option git config --local index.skipHash false. then you have to clear your index and restage all your changes so that the index gets built with the option disabled.

I expect we'll have to wait for libgit2 to fix it and for the changes to propagate downstream.

[chriskrycho]

Bonus note: you may not be setting index.skipHash directly. If you set feature.manyFiles = true, that will have the same effect (it implicitly enables index.skipHash, I believe).